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ABSTRACT 


This  thesis  examines  the  potential  effect  of  unmanned  aerial  vehicles  (UAV)  in 
the  Decisive  Stonn  operations.  This  analysis  of  UAV  technology  will  assist  Gulf 
Coalition  Forces  decision  makers  in  their  selection  of  the  most  suitable  and  cost-effective 
unmanned  aerial  vehicles  to  support  detection  operations.  We  use  Map  Aware  Non- 
Uniform  Automata,  an  agent-based  simulation  software  platform,  for  the  computational 
experiments.  It  models  the  operational  area,  system  entities  for  the  Gulf  Coalition 
Forces,  and  the  Houthi  militia’s  attempts  to  cross  the  Saudi-Yemeni  border.  The 
software  collects  relevant  data  that  can  be  translated  into  measures  of  mission 
effectiveness.  Results  from  10,400  simulation  runs  of  Houthi  efforts  to  infiltrate  the 
operational  area  are  analyzed  using  descriptive  statistics,  linear  regression,  and 
partition  trees.  These  results,  which  include  a  significantly  increased  percentage  of 
infiltrators  being  detected  by  Gulf  Coalition  Forces  and  improved  time  to  detect 
them,  support  the  use  of  UAVs  in  detection  missions.  Computer  experimentations 
and  analyses  reveal  the  most  significant  capabilities  the  UAV  should  have 
to  achieve  operational  goals.  These  significant  capabilities  fall  within  the  U.S. 
Department  of  Defense’s  Group  3  UAV  classification.  Therefore,  the  Gulf  Coalition 
Forces  should  consider  procuring  UAVs  in  that  category  to  enhance  detection  in  the 
operational  area. 
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EXECUTIVE  SUMMARY 


Saudi  Arabia  leads  the  Gulf  Coalition  Force  (Bahrain,  United  Arab  Emirates, 
Qatar,  and  Kuwait)  in  executing  Operation  Decisive  Storm,  a  military  operation  in 
Yemen  against  Houthi  militias  and  their  allies  (Adaki  2015;  Mello  and  Knights  2016). 
This  operation  defends  the  Yemeni  government  and  protects  the  borders  of  northern 
Yemen’s  and  the  Kingdom  of  Saudi  Arabia  ( Al  Jazeera  2016). 

Detecting  the  militia’s  scouts,  infiltrators,  and  terrorists  before  they  perform  their 
reconnaissance  or  attack  is  critical  for  defending  the  border  and  Gulf  Coalition  personnel. 
David  Roberts  (2012)  notes  that  during  the  Gulf  Coalition  Force  (GCF)  operation  in 
Yemen,  the  Houthi  militias  successfully  gathered  location  information  about  the  Emirati 
and  Bahraini  troops.  With  this  information,  the  Houthi  militias  were  able  to  fire  a  Soviet 
Tochka  missile  that  killed  more  than  50  soldiers.  Other  attacks  have  also  resulted  in 
losses  to  the  GCF,  frequently  because  of  the  force’s  inability  to  detect  those  infiltrators. 

This  research  examines  the  potential  effects  of  employing  unmanned  aerial 
vehicles  (UAV)  in  the  Decisive  Stonn  Operations  in  Yemen.  Results  will  assist  decision 
makers  in  their  acquisition  selection  of  an  appropriate  UAV  to  support  detection 
operations.  Furthennore,  this  study  shows  how  the  GCF  participating  in  Yemeni 
operations  can  improve  border  security  and  defend  their  operational  areas  against  enemy 
incursions  and  reconnaissance.  This  thesis  analyzes  the  use  of  the  UAVs  in  conjunction 
with  detection  methods  already  being  employed  by  ground  forces  for  the  GCF’s  Decisive 
Storm  operational  area.  Using  the  UAVs  in  the  Decisive  Stonn  operational  area  might 
increase  the  probability  of  detection  of  those  infiltrators  and  militia  members. 

This  thesis  uses  Map  Aware  Non-Uniform  Automata  (MANA),  an  agent-based 
simulation  software  platform.  First,  we  examine  the  effect  of  introducing  UAV 
technology  into  the  detection  operations.  Applying  computer  experimentation  to  model 
the  Houthi  militia’s  attempts  to  cross  the  border  provides  insights  into  the  Gulf  Coalition 
Forces’  intelligence,  surveillance,  and  reconnaissance  (ISR)  capabilities.  Additional 
experimental  designs  effectively  explore  the  required  UAV  characteristics  that  can 
enhance  the  detection  mission.  Regression  analysis  and  partition  tree  analysis  provide 

xvii 


additional  assistance  in  examining  all  design  point  options.  Finally,  a  relative  cost  and 
benefit  analysis  helps  in  identifying  the  design  option  we  wish  to  pursue. 

The  introduction  of  unmanned  aerial  vehicles  supporting  the  detection  missions  in 
the  Decisive  Storm  operation  reveals  significant  improvements  in  the  percentage  of 
infiltrators  that  the  Gulf  Coalitions  Forces  detect  as  well  as  in  the  time  it  takes  to  detect 
them.  Nevertheless,  in  the  first  introduction,  the  UAVs  lacked  the  necessary  capabilities 
and  thus  were  unable  to  support  GCF  detection  operations.  Therefore,  we  analyzed  more 
computer  experimentation  to  discover  the  UAV  characteristics  necessary  to  satisfy  the 
GCF  operational  goals.  Using  regression  analysis  we  sought  to  uncover  the  most 
significant  factors  having  an  impact  on  the  UAV’s  detection  capabilities;  this  analysis 
indicated  six  such  factors.  Furthermore,  the  partition  tree  analysis  identified  25  design 
point  options,  among  260  examined,  that  met  the  operational  goals  when  the  stealth  level 
of  the  enemy  is  low.  By  contrast,  we  found  only  seven  design  point  options  meet  the 
operational  goals  when  the  enemy  stealth  level  is  high. 

Finally,  we  calculated  a  relative  cost  estimate  for  the  total  system  price  for  all  260 
design  point  options.  The  most  desirable  option  is  revealed  in  the  relative  cost  efficiency 
analysis. 
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I. 


INTRODUCTION 


A.  PURPOSE 

This  research  examines  the  potential  effect  of  employing  Unmanned  Aerial 
Vehicles  (UAV)  in  a  detection  operation  in  the  Decisive  Storm  operation  in  Yemen. 
Results  will  assist  decision  makers  in  their  acquisition  selection  of  an  appropriate  UAV  to 
support  detection  operations.  Furthennore,  this  study  can  show  how  the  Gulf  Coalition 
Forces  (GCF)  participating  in  the  Decisive  Storm  operations  can  improve  border  security 
and  defend  their  operational  areas  against  enemy  incursions  and  reconnaissance. 

B.  MOTIVATION 

Saudi  Arabia  leads  the  Gulf  Coalition  Force  (Bahrain,  United  Arab  Emirates, 
Qatar,  and  Kuwait)  in  executing  Operation  Decisive  Storm,  a  military  operation  in 
Yemen  against  Houthi  militias  and  their  allies  (Adaki  2015;  Mello  and  Knights  2016). 
This  operation  defends  the  Yemeni  government  and  protects  northern  Yemen’s  and  the 
Kingdom  of  Saudi  Arabia’s  borders  ( Al  Jazeera  2016). 

Detecting  the  militia’s  scouts,  infiltrators,  and  terrorists  before  they  perform  their 
reconnaissance  or  attack  is  critical  for  defending  the  border  and  the  Gulf  Coalition 
personnel.  Roberts  (2012)  stated  that  during  the  GCF  operation  in  Yemen,  the  Houthi 
militias  successfully  gathered  location  information  about  the  Emirati  and  Bahraini  troops. 
With  this  information,  the  Houthi  militias  were  able  to  fire  a  Soviet  Tochka  missile  that 
killed  more  than  50  soldiers.  Other  occasions  have  also  resulted  in  losses  to  the  GCF, 
frequently  because  of  the  force’s  inability  to  detect  those  infiltrators. 

This  chapter  provides  infonnation  about  the  Gulf  Corporation  Council  (GCC)  and 

its  integrated  military  forces,  the  Peninsula  Shield  Forces  (PSF),  for  which  the  Unmanned 

Aerial  System  (UAS)  is  a  potential  solution.  A  short  background  about  the  conflict 

environment  and  the  threats  confronting  the  Kingdom  of  Saudi  Arabia’s  border  and  on  all 

other  GCC  members  follows  this  discussion.  This  chapter  lists  major  challenges 

in  securing  the  Yemeni-Saudi  border,  describes  the  inadequate  detection  means  used  by 

the  GCF  in  that  region,  and  suggests  probable  solutions. 
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c. 


BACKGROUND 


This  section  provides  essential  information  about  the  GCC  and  the  conflict 
environment.  It  discusses  the  Decisive  Storm  objectives  and  challenges.  This  section 
further  explains  the  current  detection  means  used  by  the  GCF  and  other  possible  options 
to  enhance  the  detection  operations. 

1.  GCC  Strategic  Location  and  Surrounding  Challenges 

The  six  GCC  countries  (Kingdom  of  Saudi  Arabia,  Kingdom  of  Bahrain,  United 
Arab  Emirates,  Oman,  Kuwait,  and  Qatar)  are  located  in  the  Arabian  Peninsula. 
Together,  these  countries  form  a  coalition  for  resolving  or  mitigating  political,  economic, 
and  security  issues.  They  have  merged  their  security  policies  and  defense  capabilities  to 
form  the  Peninsula  Shield  Forces  (PSF).  The  GCC  states  share  borders  with  Yemen  in  the 
southern  part  of  the  peninsula  and  Iraq  and  Jordan  in  the  northern  part  (Figure  1). 
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State  of  Kuwait 


Kingdom  of  Bahrain 


Kingdom  of  Saudi  Arabia 
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Figure  1 .  The  Gulf  Corporation  Council  States.  Adapted  from 
Secretariat  General  of  the  GCC  (2017). 


The  GCC  countries  are  surrounded  by  countries  that  have  internal  conflicts  and 

elements  of  destabilization  making  it  critical  for  the  GCC  states  to  protect  their  borders 

against  terrorists,  spies,  members  of  criminal  organizations,  and  illegal  immigrants. 
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Investing  in  detection  and  intelligence  equipment  is  essential  to  maximize  border 
protection.  In  particular,  rapid  technological  changes  and  evolving  concealment 
techniques  used  by  terrorists  demand  new  means  for  detection  and  intelligence  gathering. 

2.  Decisive  Storm  Objectives 

In  March  2015,  Saudi  Arabia  formed  the  Gulf  Coalition  to  defeat  the  Houthi 
militias  and  those  loyal  to  the  former  president  of  Yemen  and  to  restore  legitimacy  of  the 
Yemeni  government.  This  quest  was  based  on  United  Nations  (UN)  Resolution  number 
2216  (UN  Security  Council  2015). 

Along  with  Gulf  States  (except  Oman),  other  nations  have  joined  the  military 
operations  (Figure  2).  The  United  States  has  joined  the  operation  and  provides  logistical 
and  intelligence  support  to  the  GCF  during  the  military  operation  in  Yemen  (Alshabeeb 
2015).  Graham  (2016)  reports  that  United  States’  (U.S.)  forces  as  well  as  the  United 
Kingdom  C4I  (command,  control,  communications,  computers,  and  intelligence) 
personnel  have  been  deployed  in  the  operation  centers  in  command  and  control  to 
provide  assistance  in  controlling  air  strikes  and  detecting  threats. 


THE  DECISIVE  STORM’COALITION 

Saudi  Arabia,  along  with  9  other  countries,  launched  a  military  operation  against  the  Houthi  militia. 

Gult  States  -  with  the  exception  ot  Oman  -  are  providing  military  support. 
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Figure  2.  Decisive  Storm  Military  Coalition  Forces. 

Source:  Alshabeeb  (2015). 
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3. 


Conflict  Environment 


After  the  Iraq  war  in  2003  and  the  Arab  Spring,  Iraq  and  Yemen  suffered  from  an 
unstable  security  environment,  as  well  as  a  weak  economy  (Cordesman,  2015).  Both 
countries  have  borders  adjacent  to  GCC  nations.  Weak  security  situations  and  availability 
of  arms  and  weapons  have  nurtured  a  fertile  enviromnent  for  terrorist  organizations  to  start 
their  operations.  Terrorist  organizations  increase  the  threat  of  smuggling  and  drug 
trafficking  as  well  as  migrating  terrorist  members  into  the  GCC  countries  to  vandalize 
civilian-  and  government-owned  properties.  Michael  Knights  (2006)  has  pointed  to  this 
issue,  stating  that  “contiguous  borders  with  Yemen  and  Iraq — both  key  theaters  of 
operation  for  jihadists — make  Saudi  Arabia  a  critical  transshipment  point  for  weaponry  and 
jihadists  engaged  in  multidirectional  flow  of  personnel  and  equipment  throughout  the 
GCC”  (35).  He  argues  that  the  threat  posed  by  terrorists  from  Yemen  and  Iraq  necessitates 
enhancing  security  for  GCC  countries  that  share  borders  with  Yemen  and  Iraq. 

4.  Decisive  Storm  Challenges 

Detecting  infiltrators  is  the  key  to  success  for  the  GCF  if  they  want  to  ensure  border 
security.  Additionally,  securing  the  operational  area  against  enemy  incursion  and 
reconnaissance  is  critical.  There  are  many  challenges  that  limit  the  detection  process  in  that 
region. 


a.  Technological  Challenge 

The  first  challenge  is  detecting  the  militia’s  scouts  or  other  infiltrators,  which  is 
critical  for  securing  the  border  and  protecting  the  Coalition  Forces.  Electronic  detection 
equipment  such  as  ground  radars  is  inadequate  because  of  the  size  of  the  operational  area 
and  the  characteristics  of  the  terrain  that  interfere  with  such  equipment.  This  leaves  the 
GCF  vulnerable  to  the  Houthi  militias,  who  have  been  sending  their  scouts  across  the 
border  to  collect  reconnaissance  information  about  the  Coalition  troops’  locations  to  attack 
and  cause  damage. 
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b.  Terrorism  Challenge 

The  GCF  must  contend  with  terrorist  organizations,  such  as  Al-Qaeda  in  the 
Arabian  Peninsula  (AQAP)  or  the  Islamic  State  in  Iraq  and  Syria  (ISIS  or  Daesh)  located 
in  Yemen  that  are  trying  to  enter  the  Arabian  Peninsula  to  cause  havoc.  Saeed  Al-Batati 
and  Kareem  Fahim  (2015)  report  that  terrorist  organizations  located  in  Yemen,  such  as 
AQAP  and  ISIS,  are  trying  to  take  advantage  of  the  instability  and  loss  of  border  control 
to  enter  the  Arabian  Peninsula  and  carry  out  their  missions.  Al-Batati  (2016)  further 
explains  that  after  the  Yemeni  troops  loyal  to  the  former  president  left  the  city  of 
Mukkala  in  Yemen,  they  created  an  opportunity  for  terrorists.  Thus,  he  notes,  Al-Qaeda 
members  promptly  took  over  the  city  knowing  that  the  GCC  forces  and  the  Coalition 
Forces  were  otherwise  occupied  in  conflict  with  the  Houthi  militias. 

c.  Terrain  Challenge 

Another  challenge  for  the  GCF  in  the  operation  in  Yemen  is  the  geography.  The 
area  of  operation,  illustrated  in  Figure  3,  is  located  in  the  southwest  part  of  the  Arabian 
Peninsula  where  hills  and  rugged  mountains  dominate.  Gulf  Coalition  Forces  are 
accustomed  to  working  in  the  eastern  part  of  the  Arabian  Peninsula  where  dry  desert  and 
coastlands  cover  most  of  the  area  where  detection  of  infiltrators  is  easier  (Hesham  2007; 
Witty  2001;  Fryberger  et  al.  1984). 


The  image  on  the  right  shows  an  enlarged  image  of  the  Yemeni-Saudi  border  and  the 
operation  area  is  shaded  in  red 


Figure  3.  The  Arabian  Peninsula  Landscape.  Adapted  from  Nielsen  (2012). 
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5.  Detection  and  Intelligence  Methods  Used  in  Decisive  Storm 
Operations  Are  Inadequate 

Manned  aircraft  patrol  the  border  area  in  this  military  operation.  They  also 
provide  intelligence,  surveillance,  and  reconnaissance  (ISR)  over  the  border  or  around  the 
operational  areas  where  troops  are  located.  The  Gulf  Coalition  ground  forces  utilize  a 
variety  of  methods  and  equipment  to  detect  (1)  terrorists  trying  to  cross  the  border  at 
Saudi  Arabia  and  (2)  Houthi  militias  gathering  information  undercover  to  attack  GCF 
troops.  Electronic  radar  sensors,  night  vision  scopes,  ground  motion  sensors,  and 
detection  monitors  have  proven  ineffective  to  detect  these  infiltrators.  While  some 
manned  aircraft  support  has  proven  successful,  the  cost  associated  with  this  method  is 
prohibitive.  The  major  reason  for  the  failure  of  other  methods  is  the  rough  topography  of 
the  operational  area. 

Rough  topography,  the  size  and  shape  of  the  operational  area  are  significant 
factors  that  limit  the  effectiveness  of  the  GCF  to  prevent  intrusions.  These  challenges 
motivate  the  employment  of  new  detection  technologies  to  bolster  GCF  capabilities. 

The  author  visualized  the  process  by  which  the  GCF  would  decide  on  what 
detection  options  are  better  suited  for  the  force.  Following  the  systems  engineering 
mindset  in  defining  the  problem  and  suggesting  a  possible  solution,  the  author  used  a 
multiple-criteria  weighted  scoring  model  with  author  derived  weights  and  scores  to 
examine  the  best  detection  method  for  the  GCF  for  use  in  the  operation  area. 

Options  were  evaluated  against  each  other  and  given  a  weighting  based  on  their 
importance.  Total  rank  score  determined  the  most  effective  option  to  be  employed  by  the 
GCF.  Table  1  shows  the  four  options  evaluated;  the  UAV,  with  the  highest  score  of 
15.55,  is  the  best  choice  for  GCF  purposes.  Based  on  this  preliminary  study  and  the 
GCF’s  needs,  the  UAV  is  the  focus  area  in  this  thesis;  further  detailed  analysis  of  UAV 
characteristics  and  cost-related  concerns  are  presented  in  subsequent  chapters. 
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Table  1.  Analysis  of  Detection  Options  for  the  Gulf  Coalition  Forces. 


Evaluation  Measures 

Alternatives  Score 

Alternatives  Weighted  Score 

(Architectural  Drivers) 

Weights 

Manned 

aircraft 

Balloons 

Ground 

forces 

UAV 

Manned 

aircraft 

Balloons 

Gound 

Forces 

UAV 

Cost 

0.8 

1 

2 

3 

4 

0.8 

1.6 

2.4 

3.2 

Detection  Range 

0.7 

4 

2 

1 

3 

2.8 

1.4 

0.7 

2.1 

Tough  terrain  employment 

0.6 

3 

1 

2 

4 

1.8 

0.6 

1.2 

2.4 

Ground  forces  integration 

0.7 

1 

3 

4 

2 

0.7 

2.1 

2.8 

1.4 

Mobility 

0.5 

2 

1 

4 

3 

1 

0.5 

2 

1.5 

Susceptibility 

0.65 

4 

1 

2 

3 

2.6 

0.65 

1.3 

1.95 

Operational  duration 

0.75 

2 

3 

1 

4 

1.5 

2.25 

0.75 

3 

Total  Weighted  Score  -> 

11.2 

9.1 

11.15 

15.55 

Overall  Rank-> 

2 

4 

3 

1 

The  motivation  to  employ  UAV  technology  in  the  operation  in  Yemen  is 
bolstered  by  anticipated  success  in  enhancing  the  detection  and  ISR  mission  against 
terrorists  and  infiltrators,  as  well  as  by  considering  other  factors  such  as  the  cost  and 
limitations  associated  with  other  detection  options. 

D.  PROBLEM  STATEMENT 

There  is  inadequate  analysis  to  support  acquisition  decisions  to  procure  UAV 
technology  that  can  significantly  improve  Gulf  Coalition  Force  capabilities  to  detect 
crossings  or  encroachments  on  the  border. 

E.  SCOPE  OF  THE  RESEARCH 

This  study  can  aid  GCF  decision  makers  in  their  acquisition  decision  for  a 
suitable  UAV  to  support  detection  missions.  Furthermore,  this  study  can  show  the 
importance  of  integrating  the  air  surveillance  UAV  with  ground  forces  and  even  the 
efficacy  of  replacing  other  detection  techniques  currently  used.  The  study  also  identifies 
the  UAV  technology  employment  options  to  serve  the  region. 

This  thesis  analyzes  the  use  of  UAVs  along  with  other  detection  methods  already 
used  by  ground  forces  for  the  GCF.  Using  the  UAVs  in  the  Decisive  Storm  operational 
area  might  increase  the  probability  of  detection  of  those  infiltrators  and  militia  members. 
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F.  RESEARCH  QUESTIONS 

This  study  addresses  the  following  major  research  question:  How  can  a  UAV 
technology  assessment  inform  the  Gulf  Coalition  Force’s  acquisition  decisions  for 
improving  its  early  warning  detection  capabilities  in  the  Decisive  Storm  operation? 

Specifically,  this  research  is  guided  by  these  questions: 

•  To  what  degree  can  UAV  technology  improve  the  detection  capability  for 
the  GCF  to  increase  border  security  and  better  defend  the  Saudi-Yemen 
border? 

•  What  UAV  characteristics  are  most  important  for  the  GCF  to  consider  in 
procuring  UAV  technology? 

•  What  combination  of  UAV  capabilities  most  effectively  increases  target 
detection  and  classification  in  the  operational  area? 

•  What  number  of  UAV  systems  is  required  to  achieve  a  certain  threshold 
for  detection  and  enhance  border  security? 

•  How  will  cost  constraints  influence  the  selection  and  eventual  purchase  of 
the  ideal  UAV? 

G.  THESIS  ORGANIZATION 

The  remainder  of  this  thesis  is  organized  as  follows:  Chapter  II  reviews  other 
related  research  that  identifies  a  similar  UAV  role  in  detection  as  well  as  agent-based 
modeling  studies.  Chapter  III  explains  the  setup  of  the  model  and  the  scenarios  chosen 
for  the  purpose  of  this  thesis.  This  discussion  includes  brief  details  about  agents  and  their 
settings  as  well  as  explanations  of  the  terrain  map  and  elevation  maps  used  in  this 
analysis.  Additionally,  Chapter  III  explains  the  design  factors  chosen  for  this  analysis  and 
method  to  explore  the  design  of  experiment  space.  Chapter  IV  analyzes  the  experiment 
results  and  reveals  the  best  UAV  characteristic  options  for  the  GCF.  Chapter  V  concludes 
the  thesis  by  answering  the  research  questions  to  illustrate  their  achievability. 


8 


II.  LITERATURE  REVIEW 


This  chapter  provides  background  information  on  UAVs  and  their  benefits  in 
modem  warfare.  Next,  it  discusses  past  studies  about  the  UAV  role  in  border  control  and 
studies  about  modeling  UAVs  to  design  a  system.  Furthermore,  it  includes  an  explanation 
and  discussion  of  the  modeling  and  simulation  techniques  that  influence  the  approach 
used  in  this  thesis. 

A.  UAV  BACKGROUND 

The  U.S.  government  is  the  main  defense  supplier  for  the  allied  Arabian  Gulf 
countries  (Serrano  and  Vats  2016).  However,  due  to  the  restrictions  on  the  sale  of  armed 
and  unarmed  UAVs  that  the  United  States  Department  of  Defense  has  imposed  on  the 
Gulf  States  (Cordesman  and  Peacock  2015),  the  Gulf  States  lack  extensive  experience 
with  this  technology. 

A  brief  summary  of  the  unmanned  aerial  vehicle  technology  will  assist  in  the 
acquisition  decision  maker’s  understanding  of  the  UAV  system.  UAVs  are  a  component 
of  the  unmanned  aircraft  system  according  to  the  U.S.  Department  of  Defense  (DOD) 
“UAS  Road  Map  for  the  UAS  from  2005  to  2030”  (Cambone  et  al.  2005).  DOD  refers  to 
the  UAS  as  the  whole  system  whose  elements  are  used  to  control  the  unmanned  aircraft. 
Examples  are  the  vehicle,  human  operator,  payload  and  other  elements.  This  thesis  limits 
its  focus  to  the  component  of  the  UAS  system  highlighted  in  Figure  4. 
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Figure  4.  Unmanned  Aircraft  System  Components. 


UASs  come  in  different  categories  for  a  variety  of  mission  types.  Charles 
Sulewski  (2005)  describes  four  different  classification  of  UASs  in  support  of  the  U.S. 
Army:  Class  1  provides  Reconnaissance,  Surveillance  and  Target  Acquisition  (RSTA) 
capabilities  for  the  platoon  level;  Class  2  provides  the  same  capability  as  Class  1  in 
addition  to  target  designation  for  the  platoon  and  company  level;  and  Class  3  improves 
upon  Class  2  systems  by  adding  a  communication  relay  facility  on  the  UAS  for  the 
combined  arms  battalion  level.  Finally,  Class  4  UASs  have  the  capability  of  flying  in 
conjunction  with  manned  aircraft  during  missions.  Table  2  shows  the  latest  UAS 
classifications  based  on  DOD  information. 


Table  2.  Current  UAS  Systems  Classified  by  U.S.  Army. 

Source:  United  States  Army  (2010). 


Category 

Size 

Take-off  weight 

Altitude 

Airspeed 

Group  1 

Small 

0-20 

<1200 

<100 

Group  2 

Medium 

21-55 

<3500 

<250 

Group  3 

Large 

<1320 

<18000 

<250 

Group  4 

Larger 

>1320 

<18000 

Any  airspeed 

Group  5 

Largest 

>1320 

>18000 

Any  airspeed 
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The  UAV’s  rapid  technological  evolution  has  changed  its  role  in  modern  warfare. 
It  is  an  important  part  of  combat  due  to  the  additional  advantage  it  adds  to  the  forces. 

B.  RESEARCH  STUDIES 

Many  research  studies  address  the  benefit  of  UAV  for  homeland  security  and 
show  the  major  role  of  the  UAV  technology  in  this  matter.  Additionally,  significant 
number  of  thesis  research  conducted  by  Naval  Postgraduate  School  (NPS)  students  about 
modeling  UAVs  to  design  a  system,  depict  the  importance  of  UAV  in  today’s  warfare. 

1.  UAV  Effectiveness  in  Border  Security 

Unmanned  aerial  vehicles  have  become  a  key  player  in  today’s  warfare  due  to  the 
advancement  in  technology.  UAVs  can  cover  a  wide  range  of  tasks  depending  on  the 
requirement  of  the  user.  UAVs  have  associated  cost  and  finances  that  stakeholders  must 
consider  relative  to  other  procurement  requirements.  However,  many  benefits  outweigh 
those  costs  when  it  comes  to  the  security  of  borders  against  terrorists. 

Christopher  Bolkom  (2004)  points  to  the  UAV  costs  and  benefits.  He  states  that 
the  UAV  could  address  shortcomings  in  border  security  surveillance  if  the  correct  sensor 
is  used;  that  is,  a  sensor  capable  of  identifying  targets  precisely.  Similarly,  UAVs  capable 
of  longer  flight  times  can  influence  border  security  as  they  can  sustain  coverage  for 
longer  periods.  He  also  points  out  the  higher  cost  associated  with  the  purchase  of  the 
UAV  in  comparison  with  manned  aircraft,  that  difference  in  cost  is  offset  by  the  UAV’s 
superior  endurance.  Bolkom  also  identifies  some  drawbacks  associated  with  UAVs,  such 
as  the  high  rate  of  accidents,  and  relatively  lower  redundancy  and  survivability  factors  in 
UAV  design  compared  to  manned  aircrafts. 

Chad  Haddal  and  Jeremiah  Gertler  (2010)  list  the  strengths  and  weaknesses  of 
deploying  UAVs  along  the  United  States  border.  They  assert  that  UAVs  could  fill  the 
detection  gap  associated  with  currently  used  surveillance  equipment.  In  addition,  UAVs 
can  act  as  range  extenders  for  border  surveillance  when  comparing  UAVs  with  traditional 
border  surveillance  techniques.  Nevertheless,  UAVs  have  some  weaknesses,  according  to 
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Haddal  and  Gertler,  associated  with  cost  of  operation,  such  as  their  high  accident  rate  and 
the  degradation  in  target  detection  under  bad  weather  conditions. 

2.  Modeling  UAVs  in  Battlefields 

Begum  Ozcan  (2013)  has  analyzed  the  use  and  effectiveness  of  UAVs  to  support 
the  security  of  Turkey’s  east  border,  adjacent  to  the  Iraqi  border,  where  terrorist  members 
locate  and  travel  between  the  two  countries.  In  her  study,  she  illustrates  the  difficulties  in 
detecting  terrorist  members  located  in  tough,  mountainous  terrain  using  regular  detection 
equipment  and  manned  aircraft.  The  UAV  can  be  adjusted  technologically  to  meet  the 
stakeholders’  requirement  for  such  a  need,  if  the  affecting  factors  can  be  determined. 
Ozcan  built  a  model  to  explore  the  best  technological  capabilities  of  a  UAV  that  can  be 
used  to  expand  coverage  along  the  Turkey-Iraqi  border.  She  examined  possible  effects 
that  UAVs  could  have  in  detecting  and  classifying  infiltrators  and  terrorist  members 
along  the  border.  Her  study  showed  that  the  use  of  a  UAV  is  very  effective  in  the 
detection  and  classification  of  terrorist  activity  in  this  challenging  region,  taking  into 
account  the  significant  technical  factors  that  affect  UAV  detection  and  classification 
performance. 

Another  study  conducted  by  Bahri  Yildiz  (2009)  focuses  on  the  importance  of 
border  protection  in  deterring  terrorist  members,  human  and  drug  trafficking,  and  illegal 
migration  by  improving  the  detection  methods  along  the  border.  In  his  study,  Tucson, 
Arizona,  was  modeled  using  Map  Aware  Non-Uniform  Automata  (MANA)  software  to 
investigate  the  added  value  of  UAVs  in  border  security  along  with  other  detection  assets, 
such  as  ground  force  agents,  security  surveillance  towers,  sensors,  predator  UAVs,  and 
communication  centers.  Yildiz  concludes  that  UAVs  could  provide  an  additional  benefit 
in  capturing  illegal  infiltrators  and  could  increase  the  security  of  the  borders.  According 
to  the  outcomes  of  his  model,  three  affecting  factors  to  improve  the  detection  means  are 
manpower  of  the  patrolling  agents,  communication  network  infrastructures,  as  well  as  the 
UAV  itself. 

Fatih  Sen  (2015)  points  to  the  challenges  that  Turkey  faces  protecting  and 
securing  the  Turkey-Iraqi  border  and  the  limitation  to  the  manned  aircraft  used  for  the 
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security  of  border  missions.  The  limited  fuel  capacity  of  manned  aircraft  and 
geographical  limitations  across  the  border  have  encouraged  the  Turkish  government  to 
explore  other  means  of  detection  to  enhance  their  ability  to  counter  terrorists  in  that 
region.  Sen  (2015)  also  uses  the  MANA  software  to  model  different  scenarios  that  are 
possible  in  that  region  and  analyzes  the  outcomes  to  test  the  use  of  UAVs  and  manned 
aircraft.  His  study  outcome  shows  that  with  the  use  of  UAVs,  additional  target 
identification  could  be  observed,  which  can  lead  to  protecting  the  borders  more 
effectively  against  terrorist  attacks. 

C.  AGENT-BASED  MODELING  SIMULATION 

Simulation  software  plays  a  major  role  in  the  analysis  of  combat  and  supporting 
decision  making,  as  it  shapes  solutions  to  the  problem  and  explores  numerous  possible 
outcomes  with  low  cost.  Several  such  applications  include  wargaming,  training,  system 
overall  efficiency  simulators,  and  modeling  software  enables  different  military  tactics  to 
be  practiced.  Thomas  Lucas  et  al.  (2012)  note  that  DOD  decision  makers  usually  use 
simulation  to  make  decisions  on  acquisition  programs,  equipment  employment  choices, 
and  possible  tactics  and  procedures. 

Eric  Bonabeau  (2002)  lists  several  advantages  of  agent-based  simulation  over 
other  techniques.  Such  advantages  include  the  ability  to 

•  capture  emergent  events  that  result  from  interactions  of  individual  entities, 

•  provide  natural  description  of  the  system  or  closer  to  reality  simulation, 

•  provide  flexibility  of  the  system  by  adding  or  subtracting  agents  or 
objects. 

Simon  Taylor  (2014)  asserts  that  “agent  based  modeling  has  evolved  as  a  natural 
response  to  the  need  for  complex  system  modeling”  (3).  He  also  mentions  that  agent- 
based  modeling  has  the  capability  to  represent  a  system  in  a  sensible  way,  more  so  than 
other  traditional  techniques.  Based  on  his  definition  of  the  agent-based  model,  each 
agent-based  model  should  have  four  characteristics: 

•  Each  agent  should  have  a  set  of  attributes  that  define  the  condition  of  the 
agent  and  his  actions. 
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•  Relationships  must  define  how  the  agents  relate  and  interact  with  other 
agents  and  the  environment  in  the  model. 

•  The  environment  where  the  agent  lives  and  variables  define  how  he  deals 
with  the  environments. 

•  A  system  consists  of  agents,  environment,  and  the  relationship  between 
them. 

The  agent-based  modeling  is  generated  using  a  suitable  programing  tool  and  can 
be  executed  through  a  wide  range  of  commercially  available  programs,  such  as  MANA, 
MATLAB,  C++,  and  other  programs. 

D.  MAP  AWARE  NON-UNIFORM  AUTOMATA  SOFTWARE 

Map  Aware  Non-Uniform  Automata,  or  MANA,  is  one  of  the  simulation 
programs  that  incorporates  agent-based  modeling.  MANA  simulation  captures  the 
unexpected  outcomes  of  interactions  between  agents  based  on  their  behavior  in  the 
simulation  environment.  Agents  could  be  in  fonn  of  a  human,  airplane,  UAV,  vehicle, 
ship,  and  other  types  of  key  players  in  the  simulations.  As  mentioned  earlier,  MANA  is 
an  agent-based  distillation  model  that  excludes  unneeded  complex  computations,  but  at 
the  same  time  produces  all  necessary  data  required  for  the  analysis.  MANA  was 
developed  to  explore  key  concepts:  (1)  situational  awareness,  (2)  communications,  (3) 
terrain  maps,  (4)  waypoints,  and  (5)  agent  personality  change  due  to  events  (McIntosh 
2007). 

Ozcan  (2013)  identifies  some  of  the  advantages  of  MANA.  It  is  easy  to  learn  in  a 
short  time  compared  with  other  modeling  software,  is  user  friendly,  and  has  a  simple 
graphical  user  interface  (GUI)  that  enables  building  scenarios  in  a  shorter  period.  Many 
other  Naval  Postgraduate  School  theses  have  used  MANA  to  analyze  UAV  capabilities 
and  effectiveness  such  as  those  by  Arif  Ipekci  (2002),  Sulewski  (2005),  Ozcan  (2013), 
Sen  (2015),  and  Whye  Kin  Melvin  Cheang  (2016). 
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III.  METHODOLOGY 


A.  INTRODUCTION 

To  address  the  research  questions  in  this  study,  we  use  a  modeling  and  simulation 
based  systems  engineering  approach.  We  use  abstractions  of  the  relevant  entities  in  the 
operational  scenario  to  conduct  simulation  experiments  that  produce  data  with  the 
appropriate  pedigree  that  enables  in-depth  analysis  and  examination  to  support 
acquisition  decisions.  This  chapter  starts  with  a  discussion  of  the  measures  of 
effectiveness  (MOE),  followed  by  a  scenario  description  for  the  situation  in  which  the 
system  will  be  placed  and  simulated  in  the  MANA  software.  Following  that  is  a 
discussion  of  significant  factors  that  might  influence  the  MOE.  Finally,  we  describe  the 
design  of  experiments  that  we  use  to  efficiently  and  effectively  explore  the  decision 
space. 

B.  MEASURE  OF  EFFECTIVENESS 

John  M.  Green  (2001)  states  that  every  system  development  process  should  have 
a  performance  analysis;  it  is  essential  to  evaluate  the  system’s  development  within  its 
resources.  Part  of  this  perfonnance  analysis  is  the  performance  measurements  that  need 
to  be  carefully  selected,  also  called  the  MOE.  Green  defines  the  MOEs  as  “the  measure 
of  effectiveness  that  provide  a  quantifiable  benchmark  against  which  the  system  concepts 
and  implementation  can  be  compared”  (1). 

The  measure  of  effectiveness  should  be  allied  with  defining  and  achieving  the 
operational  concerns.  This  thesis  embraces  two  MOEs  that  are  related  to  the  improved 
effectiveness  of  the  GCF’s  detection  capabilities  when  UAVs  are  integrated  with  ground 
radars.  Those  two  MOEs  will  aid  in  answering  the  research  questions  of  this  study. 
Simulations  have  red  and  blue  agents  to  describe  different  factions  in  the  operational 
environment. 

The  first  MOE  is  the  percentage  of  the  total  red  agents  detected  (MOE  1).  This 

percentage  is  obtained  at  the  end  of  each  run.  Each  simulation  run  has  500  red  agents 

trying  to  cross  through  the  military  restricted  area.  MOE  1  represents  the  added  benefits 
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of  employing  a  UAV  in  conjunction  with  the  available  ground  radars.  Detecting  100%  of 
the  red  agents  is  the  ultimate  goal  of  the  GCF  commanders,  but  it  is  unrealistic. 
Therefore,  we  set  the  MOE  1  threshold  to  90%  of  the  red  agents,  which  equates  to  450 
red  agents. 

The  second  MOE  is  the  time  it  takes  to  detect  40%  of  the  red  agents  (MOE  2). 
Detecting  red  agents  at  the  early  stages  of  the  operation  is  important  so  that  the  GCF  can 
determine  the  center  of  activity  to  pursue.  The  mountainous  environment  that  the  GCF 
must  traverse  to  engage  infiltrators  makes  timely  detection  a  necessity  to  rally  a  timely 
response.  As  mentioned  in  earlier  chapters,  Houthi  militias  (red  agents)  are  adept  in 
rugged  terrain  and  travel  faster,  which  requires  the  GCF  to  respond  quickly.  The  total 
simulation  run  time  is  24  hours.  We  set  the  MOE  2  threshold  to  3.55  hours  or  15%  of  the 
total  simulation  time. 

C.  SCENARIO  DESCRIPTION 

The  scenario  in  this  model  is  based  on  the  historical  attacks  by  Houthi  militias 
against  the  GCF  that  have  resulted  in  the  deaths  of  Saudi,  Bahraini  and  Emirati  troops. 
Houthis’  reconnaissance  missions  are  successful  due  to  the  weak  detection  equipment 
available  to  the  GCF.  Additionally,  Houthis  have  the  advantage  of  using  the  rugged 
terrain  to  conceal  their  movement  when  gathering  infonnation. 

In  the  scenario,  blue  forces  represent  the  GCF,  who  try  to  defend  against  the 
enemy  and  detect  them  before  they  reach  their  goals.  The  GCF  have  radars  across  the 
Saudi-Yemeni  border  to  detect  infiltrators  before  they  cross  the  border  and  reach  their 
objectives.  However,  ground  radars  have  a  low  probability  of  detecting  the  enemy 
because  of  the  terrain  and  the  enemy’s  ability  to  avoid  detection. 

The  red  forces  are  the  Houthi  militias  who  try  to  make  their  way  to  the  border  to 
perform  reconnaissance  without  being  detected  by  blue  force  detection  equipment.  As 
previously  noted,  there  are  500  red  agents  in  this  scenario. 

We  are  examining  the  improvement  in  detecting  infiltrators  when  UAVs  are 
employed  by  the  GCF.  For  this  reason,  we  initially  model  a  baseline  configuration  that 
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represents  the  current  detection  equipment  used  by  the  GCF.  Later  on,  we  add  UAVs  to 
examine  improvement  in  detection.  Data  is  collected  at  the  end  of  each  configuration  run 
to  develop  an  analysis  to  answer  the  thesis  questions. 

1.  Baseline  Configuration  (Ground  Radars  Only — Current  Capabilities) 

The  baseline  model  configuration  simulates  the  current  situation  of  the  GCF 
method  explained  previously.  Five  hundred  red  agents  divided  into  ten  groups  of  50  each, 
are  trying  to  cross  the  border  successfully  without  being  detected  in  the  military  restricted 
area.  The  GCF  ground  radars  are  the  only  detection  equipment  used  in  this  configuration. 
The  baseline  configuration  results  will  be  used  for  comparison  and  analysis  against  the 
improved  configuration. 

The  appearance  of  any  agent  in  the  restricted  area  is  considered  a  target  with 
which  GCF  must  deal.  It  is  a  restricted  area  and  presence  in  this  region  is  prohibited. 
Figure  7  depicts  the  battlefield  locations.  If  a  red  agent  is  detected  by  any  of  the  radars,  it 
is  reported  to  the  GCF  and  eliminated  from  the  scenario.  Figure  5  shows  the  locations  of 
the  red  agents,  ground  radars,  and  borderline  in  the  baseline  configuration. 
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2.  Establishing  the  Capabilities  of  the  GCF  Radars 

Because  the  number  of  radars  used  across  the  border  by  the  GCF,  the  probability 
of  detection  by  the  radars,  and  the  radar  locations  are  all  classified,  we  implement  a 
reverse  engineering  approach  to  build  an  initial  model  that  reasonably  represents  the  GCF 
detection  capability  with  baseline  radars. 

Multiple  combinations  of  the  number  of  radars,  radar  ranges,  and  radar  times 
between  detection  were  tested  in  the  model.  These  combinations  resulted  in  a  percent  of 
red  agents  detected.  To  establish  a  baseline  configuration,  the  author  used  the 
combination  of  value  settings  of  the  number  of  radars,  radar  ranges,  and  radar  times 
between  detections  that  resulted  in  approximately  30%  of  red  agents  being  detected.  This 
percentage  is  the  author’s  best  estimate  based  on  his  operational  experience  and  an 

understanding  of  the  GCF  policies.  More  details  are  provided  in  Chapter  IV. 
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3.  Improved  Configuration  (Ground  Radars  and  UAVs) 

The  improved  configuration  is  similar  to  the  baseline  configuration  except  for  the 
employment  of  the  UAVs  to  increase  the  detection  rate.  UAVs  will  patrol  the  area  of 
responsibility  (AOR)  or  the  restricted  area  to  detect  red  agents.  This  configuration  will 
employ  different  numbers  of  UAVs  (from  one  to  four).  As  the  number  of  UAVs 
increases,  their  search  pattern  and  the  area  of  coverage  change  in  that  particular  scenario. 
Results  of  the  baseline  configuration  will  be  compared  to  the  improved  configuration  to 
address  the  research  questions.  Figure  6  shows  a  screenshot  of  the  improved 
configuration  with  four  UAVs;  the  UAVs  are  circled  in  the  figure. 


Figure  6.  Improved  Configuration  Snap  Shot  with 
Four  UAVs  Setup. 
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4.  Summary  of  Scenario  Description 

Three  different  groups  of  entities  are  modeled  in  this  thesis.  These  groups  are  red 
agents  that  represent  the  enemy;  blue  radars,  which  represent  the  current  detection 
equipment  used  by  the  GCF;  and  finally,  the  blue  UAVs,  which  represent  the  new 
detection  means.  There  will  not  be  any  simulated  combat  in  this  study. 

a.  Red  Agents 

Red  agents  are  Houthi  militias  and  other  terrorist  group  members  trying  to  cross 
the  border;  they  consist  of  ten  groups  of  50  members  in  each  group.  Their  main  objective 
is  to  reach  the  border  and  get  close  to  the  GCF.  The  red  agents  are  located  at  the 
southern  part  of  the  operational  area  and  are  moving  north  towards  Saudi- Yemeni 
border,  at  a  walking  speed  of  four  kilometers  per  hour. 

Red  agents  have  a  concealment  level  that  allows  them  to  hide  from  blue  detection 
assets.  The  higher  the  concealment  level,  the  better  the  red  agents  are  at  staying  away 
from  detectors.  In  addition,  they  can  increase  their  camouflage  by  using  terrain  paths,  for 
instance,  by  traveling  off  roads  or  in  mountains  that  have  higher  concealment  levels  than 
roads.  Blue  force  radars  along  the  border  or  the  UAVs  can  detect  red  agents  at  any  time 
during  the  operation.  When  red  agents  are  detected,  detection  timing  information  will  be 
recorded. 

b.  Blue  Agents 

Blue  radars  across  the  border  have  maximum  range  and  detection  capabilities. 
Nonetheless,  some  of  the  radars  have  limited  detection  because  of  the  topography  of  the 
scenario,  which  may  prevent  blue  radars  from  detecting  red  agents.  Blue  UAVs  will 
jointly  perform  detection  of  red  agents  along  with  blue  radars.  Each  simulation  run 
represents  24  hours  of  the  operation. 

D.  DESIGN  FACTORS  AND  EXPERIMENTAL  DESIGN 

Designs  of  experiment  (DOE)  are  used  to  examine  the  relationship  and  effects  of 
different  factors  that  might  influence  the  outcomes  of  the  MOE.  Every  factor  in  the  DOE 
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has  a  range  of  values.  The  author  has  used  analogous  operational  systems  to  estimate  the 
appropriate  range  of  values  used  in  the  DOE.  In  this  thesis,  a  DOE  helps  to  identify 
factors  that  have  the  largest  effect  on  the  MOE.  The  DOE  further  assists  in  reducing  the 
variance  of  MOE  estimates. 

Jack  Kleijnen  et  al.  (2005)  describe  the  DOE  as  a  method  that  permits  the  user  to 
vary  input  parameters  to  produce  multiple  variations  in  the  model  in  an  effective  way. 
The  DOE  technique  is  used  to  detennine  the  cause  and  effects  bond,  which  is  necessary 
for  the  selection  of  the  inputs  to  enhance  outcomes.  Douglas  Montgomery  (2008)  states 
that  a  design  of  experiment  is  a  test  where  changes  are  made  to  the  input  variables  on 
purpose,  so  we  can  observe  the  reasons  for  a  change  in  the  outcomes. 

Factors  can  be  categorized  as  controllable  and  uncontrollable  factors.  Controllable 
factors  are  those  elements  of  the  system  that  can  actually  be  changed  by  the  manufacturer 
or  operators.  The  maximum  flight  altitude  or  speed,  for  example,  are  controllable  factors. 
Uncontrollable  factors  are  the  factors  that  the  designer  cannot  change  in  real  life  but  can 
change  during  the  experiment.  Enemy  characteristics  and  numbers  are  some  examples  of 
uncontrollable  factors. 

1.  UAV 

The  UAV  factors  were  selected  based  on  Group  1  to  Group  3  UAV  technical 
characteristics  (see  Table  2),  since  this  type  of  UAV  category  fits  the  required  mission  in 
detection  and  border  control  (United  States  Army  2010).  Other  UAV  groups  were 
eliminated  for  two  reasons:  they  fly  at  a  strategic  high  altitude  level  that  is  not  required 
for  this  research,  and  their  operating  radius  is  very  large. 

The  number  of  UAVs  (one  to  four  UAVs)  is  a  design  factor  that  will  be  crossed 
with  the  base  experimental  design.  The  UAV  number  factor  will  detennine  the  minimum 
number  of  UAVs  needed  to  achieve  the  required  thresholds  in  the  operation  area. 

•  UAV  speed  is  a  flight  performance  factor  that  applies  to  the  UAV.  The 
UAV  speed  refers  to  the  velocity  the  UAV  travels  at  during  the  search 
mission. 
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•  UAV  altitude  is  another  flight  perfonnance  factor  that  defines  the  height 
that  the  UAV  can  fly  in  the  covered  area. 

•  Endurance  is  the  ability  of  the  UAV  to  fly  without  the  need  to  refuel.  It 
defines  the  UAV’s  stamina. 

•  UAV  detection  range  is  a  sensor  related  factor  that  defines  the  maximum 
distance  at  which  the  sensor  can  detect  targets. 

•  Time  between  detections  is  a  factor  that  sets  the  period  between  “looks”  at 
a  search  area.  It  defines  how  the  UAV  detects  an  object  in  the  search  area. 

•  Slew  rate  is  the  factor  associated  with  how  many  degrees  the  UAV  sensor 
can  rotate  per  second. 

•  Aperture  is  the  angle  width  that  the  sensor  can  search  at  each  point  of  the 
rotation. 

•  UAV  refuel  time  is  the  time  it  takes  the  UAV  to  be  refueled  and  prepared 
for  flight.  This  factor  should  reflect  lower  values,  as  it  will  have  a  negative 
effect  on  the  availability  of  the  UAV.  The  more  time  UAVs  spend 
refueling,  the  higher  their  absence  rate. 

2.  Red  Agent 

Red  stealth  is  an  uncontrollable  factor.  It  represents  the  ability  of  the  red  agent  to 
avoid  being  detected  by  radars  or  UAVs.  Uncontrollable  factors  are  introduced  into 
models  to  reflect  a  real-life  source  of  variations,  which  increases  the  credibility  and 
robustness  of  outcomes  of  this  study. 

3.  Factor  Ranges 

Factor  ranges  have  been  defined  based  on  the  possible  UAV  types  that  could  be 
employed  in  the  Decisive  Stonn  operation  in  Yemen  by  the  GCF  (United  States  Army 
2010).  Table  3  presents  the  factor  ranges  chosen  for  the  experiment  that  are  defined  as 
significant  factors  and  might  have  impact  on  the  outcomes  of  the  research.  All  of  the 
factors  were  chosen  based  on  UAVs  described  in  open  sources. 
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Table  3.  Design  of  Experiment  Factors  and  Ranges. 


Agent 

DOE  factor 

Min. 

Max. 

Units 

UAV 

Speed 

80 

200 

Knots 

UAV 

Altitude 

3000 

8000 

Meters 

UAV 

Maximum  detection  range 

1000 

4000 

Meters 

UAV 

Time  between  detection 

5 

20 

Seconds 

UAV 

Endurance 

1 

8 

Hours 

UAV 

Refuel  time 

30 

60 

Minutes 

UAV 

Slew  rate 

60 

360 

Degrees 

UAV 

Aperture 

60 

120 

Degrees 

Red  forces 

Red  stealth 

65 

85 

Percentage 

UAV 

Number  of  UAVs 

1 

4 

Discrete 

E.  PREPARING  MANA  SOFTWARE  FOR  SIMULATION 

MANA  software  is  used  as  the  simulation  modeling  software  due  to  the  ease  with 
which  the  user  can  create  scenarios  and  modify  agents’  properties.  MANA  has  the  ability 
to  model  the  behaviors  of  the  agents  to  a  higher  resolution.  Another  reason  to  use  MANA 
is  to  take  advantage  of  the  Simulation  Experiments  and  Efficient  Designs  (SEED)  center 
at  the  Naval  Postgraduate  School  (NPS),  which  offers  scenario  development  support  and 
data  analysis,  as  well  as  a  power-computing  cluster  that  can  perfonn  complex  and 
multiple  factor  levels  simulations. 

1.  Data  Entry  and  Control 

A  database  is  created  in  MANA  to  store  entities’  values  required  for  the  scenario. 
These  values  control  the  behavior  of  and  set  the  limitations  for  entities  in  the  simulation. 
MANA  has  the  ability  to  store  different  types  of  maps  to  define  the  agent’s  movement  in 
the  simulation. 

2.  Battlefield 

An  area  of  90  x  90  km  along  the  Saudi- Yemeni  border  was  chosen  to  represent 
the  operational  area.  This  area  was  modeled  in  MANA,  with  relevant  entities,  to 
represent  the  GCF  military  operations  in  Yemen.  The  battlefield  area  is  a  restricted 
military  zone  and  divided  into  two  sections.  The  northern  part  is  the  Kingdom  of  Saudi 
Arabia,  where  the 
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GCF  are  located.  The  southern  part  is  a  rugged  area  where  Houthi  militias  and  terrorist 
group  members  make  their  way  towards  the  border.  As  previously  mentioned,  the 
military  restricted  area  excludes  not  only  military  personnel,  but  also  civilians  or  neutral 
personnel.  Any  detection  in  this  area  will  be  considered  a  target  that  the  GCF  will  treat  as 
a  threat. 

The  border  is  defended  by  multiple  radar  systems  to  detect  infiltrators  and  report 
the  detection  back  to  the  GCF  operation  centers  to  deal  with  the  infiltrators.  Figure  7 
represents  the  battlefield  agent’s  locations  and  the  border  area. 


Figure  7.  Battlefield  Description  and  Entities  Locations. 


3.  Map  Construction 

MANA  can  store  elevation,  terrain,  and  background  maps  to  define  the  agent 
movement  in  the  scenario.  The  difference  in  elevation  heights  can  act  as  a  concealment 
factor  for  the  agent  that  increases  the  stealth  of  the  agent,  or  restrict  the  movement  of  the 
agent  towards  its  goal.  It  also  plays  a  role  in  the  line  of  sight  (LOS)  factor  for  the  sensors 
that  may  delay  the  detection  time. 
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The  terrain  features  map  was  constructed  in  MANA  to  reference  the  elevation 
areas.  It  consists  of  three  different  levels:  mountains,  off  roads,  and  valleys.  To  ensure  the 
ground  radars  and  UAV  sensors  operate  properly,  it  is  imperative  to  define  these  levels  in 
the  model.  Figure  8  shows  the  terrain  map  with  the  terrain  characteristics  levels  of  going, 
cover,  and  concealment,  as  well  as  the  terrain  surface  levels  of  road,  mountain,  and  off¬ 
road.  “Going”  refers  to  the  level  of  movement;  “Conceal”  is  the  visibility  level;  and 
“Cover”  is  the  protection  level  the  agent  could  have.  Those  three  levels  range  from  zero 
to  one.  Zero  means  “No”  and  one  means  “Full.”  For  example,  when  an  agent  uses  the 
“Road,”  he  has  ease  of  movement,  but  no  cover  or  concealment  against  detection.  While 
in  the  mountains,  the  agent  has  a  high  level  of  concealment  or  a  good  chance  to  avoid 
detection,  but  difficulty  in  movement. 


Figure  8.  Terrain  Features  Map  and  Parameters. 
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F.  DESIGN  OF  EXPERIMENT 

Examining  all  factors  listed  in  Table  3  requires  a  time-efficient  design  of 
experiment  technique  to  cover  all  factors  and  to  effectively  and  uniformly  sample  the 
design  space.  A  full  factorial  experiment  is  one  option  that  requires  an  extended  period  to 
complete.  A  full  factorial  design  for  this  thesis  will  produce  170  million  design  points, 
which  would  require  more  than  five  years  to  process  if  each  run  took  one  second  to 
complete.  While  the  author  desires  a  thorough  exploration  of  design  space  for  this 
research  to  assure  discovering  important  insights,  this  amount  of  time  is  impractical. 
Susan  Sanchez  and  Hong  Wan  (2015)  explain  that  experiments  should  be  designed  in  a 
keen  manner  by  using  a  well-organized  space  filling  design,  which  this  research 
accomplishes,  as  the  following  discussion  describes. 

1.  Nearly  Orthogonal  Latin  Hypercube  Design  Method 

In  lieu  of  a  full  factorial  design,  we  are  going  to  use  the  nearly  orthogonal  Latin 
hypercube  (NOLH)  65-design  point  method  developed  by  Thomas  Cioppa  and  Thomas 
Lucas  (2007).  NOLH  design  is  a  method  used  to  combine  orthogonal  and  uniform 
designs  to  create  efficient,  uncorrelated,  fully  space-filling  experiments  (Cioppa  and 
Lucas,  2007).  It  significantly  decreases  the  number  of  design  points  (DP)  in  the 
experiment  compared  to  a  full  factorial  design.  For  these  reasons,  we  chose  an  NOLH 
design.  This  type  of  design  also  works  best  when  design  factors  are  continuous  values 
such  as  those  listed  in  Table  3. 

An  NOLH  spreadsheet  created  by  the  NPS  SEED  center  (n.d.)  was  used  to 
generate  a  unique  combination  of  factor  levels  for  this  thesis.  A  65-run  NOLH  design  to 
explore  the  factors  in  Table  3  provides  good  coverage  of  the  possible  combinations  in  the 
input  space.  Crossing  the  65  DP  with  the  number  of  UAVs  in  the  improved  configuration 
yields  260  different  design  points  (see  Appendix  A). 

The  scatter  plot  in  Figure  9  shows  the  pairwise  plots  between  factors  in  the 
experimental  design.  It  shows  the  space-filling  attributes  of  the  NOLH  in  the  two- 
dimensional  space.  If  it  were  possible  to  create  plots  for  all  other  dimensions,  they  would 
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present  similar  graphics.  They  depict  a  design  that  effectively  and  efficiently  samples  the 
problem  space. 


Figure  9.  Scatterplot  Matrix  All  Factors. 


2.  Number  of  Replications 

Each  design  point  has  to  be  replicated  to  reduce  the  variability  of  the  estimate  of 
the  MOE’s  outcome.  Replication  deals  with  the  random  errors  in  the  experiments.  For  the 
purpose  of  this  thesis,  we  have  selected  a  95%  confidence  interval  of  the  estimated  mean 
for  the  MOEs.  This  value  of  confidence  interval  was  chosen  to  create  the  desired  interval 
width  of  approximately  three  agents  for  MOE  1  and  approximately  eight  minutes  for 
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MOE  2.  From  the  operations  perspective,  the  author  asserts  that  these  are  within  the 
margins  that  the  GCF  can  effectively  operate.  Therefore,  determining  the  required 
number  of  run  replications  is  important  if  we  are  to  reach  the  desired  degree  of  precision 
in  the  results. 

We  determined  the  number  of  replications  according  to  the  formula  that  takes  into 
account  our  estimate  of  the  standard  deviation  (a)  and  our  chosen  level  of  alpha  (a  = 
0.05).  Z  is  the  variant  corresponding  to  half  of  alpha,  based  on  the  standard  normal 
distribution.  The  desired  width  of  the  confidence  interval  for  a  probability  is  w. 


n  = 


V  w  J 


Forty  replications  per  design  point  were  sufficient  for  both  MOEs.  MOE1  needed 
33  replications  while  MOE2  required  40  replications.  Therefore,  we  chose  to  run  each 
design  point  40  times.  Additionally,  this  number  of  replications  allows  the  experiment  to 
finish  in  a  reasonable  time.  The  mean  of  each  40  replications  is  ultimately  summarized 
and  used  for  the  data  analysis  (see  Appendix  B). 
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IV.  DATA  ANALYSIS 


This  chapter  contains  results  answering  the  research  questions  and  illustrating 
how  UAV  technology  can  improve  the  detection  capability  for  the  GCF.  Sensitivity 
analysis  results  are  used  to  examine  the  model  and  detennine  whether  the  system  is 
sensitive  to  scenario  parameters  such  as  the  specific  number  of  red  agents  trying  to  cross 
the  border.  Next,  a  baseline  configuration  analysis  demonstrates  the  percentage  of 
detected  red  agents  the  current  system  provides  in  the  operation.  Third,  scenario  analysis 
and  comparison  with  the  baseline  results  test  whether  UAVs  will  provide  an  additional 
percentage  of  red  agent  detection.  Fourth,  a  regression  analysis  allows  us  to  understand 
which  factors  are  significant.  Finally,  relative  cost  and  benefit  analyses  provide  an 
understanding  of  the  relative  cost  of  various  options. 

A.  SENSITIVITY  ANALYSIS  TO  RED  AGENT  SCENARIO 

The  purpose  of  the  sensitivity  analysis  is  to  examine  the  consistency  of  the  model 
outcomes  with  different  attack  scenarios  in  place.  Figure  10  shows  the  distribution  of  the 
total  number  of  red  agents  detected,  percentage  of  red  agents  detected,  and  time  to  detect 
40%  of  red  agent  using  the  improved  configuration  with  four  UAVs.  Two  different 
quantities  of  red  agents,  50  and  500,  have  been  used  to  compare  the  distribution  between 
the  two  formations. 

Results  show  that  the  average  number  of  red  agents  detected  using  the  500-agent 
formation  equals  406,  while  the  50-agent  formation  average  scored  41.  Second,  the 
average  percentage  of  red  agents  detected  for  the  500  fonnation  was  81%  with  a  1% 
increase  in  the  50-agent  formation.  Finally,  the  average  time  to  detect  40%  of  the  red 
agents  was  6.8  hours  for  the  500-agent  formation,  while  the  50-agent  formation  took 
seven  hours. 

Results  show  that  distribution  and  summary  statistics  are  very  similar  for  both 
scenario  configurations  and  the  outcomes  are  not  contingent  on  a  specific  scenario  or  a 
certain  number  of  red  agents.  These  statistics  increase  the  author’s  confidence  in  the 
model  results  and  potential  biases  in  the  data. 
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Figure  10.  Sensitivity  Model  Analysis. 


B.  RESULTS  OF  BASELINE  CONFIGURATION  (GROUND  RADARS 

ONLY) 

1.  Desired  Percentage  of  Red  Agents  Detected 

To  define  the  GCF  baseline  capability,  we  are  trying  to  find  the  right  combination 
of  ground  radars  such  that  we  detect  only  30%  of  the  red  agents.  Three  radar  factors  are 
examined  using  an  NOLH  method  to  determine  the  combination  of  the  blue  radar  factors 
that  can  achieve  the  desired  percentage  of  red  agents:  Number  of  radars,  Maximum 
detection  range,  and  Time  between  detection. 

Sixty-five  different  combinations  across  the  three  factors  have  resulted  in  the 
material  depicted  in  Appendix  C.  The  results  show  that  combinations  with  a  low  number 
of  radar  stations  (from  four  to  six)  were  not  able  to  provide  the  desired  percentage  of  red 
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agents  detected.  Therefore,  they  are  disregarded.  Combinations  with  seven  or  eight  radars 
were  able  to  generate  results  approximating  the  desired  percentage  of  red  agents  detected. 

The  highlighted  design  factor  (No#  of  radars=8,  Time  b/w  detection=108,  and 
max  range  of  5625)  came  closest  to  the  desired  baseline  percentage  of  red  agent 
detection,  registering  a  score  of  30.35%  (Table  4).  We  have  rounded  them  up  to  (No#  of 
radars=8,  Time  b/w  detection=100  seconds,  and  max  range  of  6000  meter)  to  ensure  it 
will  generate  detection  of  at  least  30%  of  the  red  agents.  Among  all  other  possible 
combinations,  we  use  those  values  as  the  baseline  for  this  study  because  those  values 
seem  to  have  the  characteristics  of  operational  understanding  of  the  situation. 


Table  4.  Blue  Radar  Design  Points  for  the  Desired  Percentage  of  Red  Agent 

Detections  (7  and  8  Radars  Only). 


NumBlueRadars 

BlueRadarRng 

BlueRadarAvgTimeBetDet 

Std  Dev(Alleg2Cas.Red.) 

Mean  (Percentage  RedDetected) 

7 

6313 

93 

8.859443752 

0.3537 

7 

5750 

83 

8.208813275 

0.348 

8 

5188 

68 

7.361542481 

0.3375 

8 

6063 

101 

8.230493645 

0.3349 

7 

7063 

115 

9.272201354 

0.33405 

8 

6625 

116 

8.738215508 

0.3301 

7 

5000 

66 

9.185237599 

0.32825 

7 

5563 

94 

7.445141248 

0.31965 

8 

5625 

108 

8.619447302 

0.3035 

8 

5438 

117 

8.556898552 

0.2872 

7 

4813 

84 

8.318615208 

0.28615 

7 

4125 

61 

9.106070109 

0.2849 

7 

4750 

92 

9.448884075 

0.28095 

7 

4188 

82 

9.508397099 

0.25945 

2.  Baseline  MOE  1  Percentage  of  Total  Red  Agents  Detected 

Figure  1 1  represents  the  distribution  of  the  percentage  of  red  agents  detected  for 
the  selected  baseline  values.  The  detection  percentage  was  attained  by  averaging  the 
outcomes  of  100  repetitions  of  the  baseline  design  point.  An  average  of  32.19%  of  red 
agent  ±  2.1%  was  detected  at  a  95%  Confidence  Interval  (Cl),  which  is  lower  than  the 
required  MOE  1  threshold  of  90%  of  red  agents  detected. 
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Figure  11.  Distribution  and  Summary  Statistics  for 
Baseline  Configuration  (MOE  1). 


C.  RESULTS  OF  THE  IMPROVED  CONFIGURATION  (GROUND  RADARS 

WITH  ONE  OR  MORE  UAVS) 

1.  MOE  1:  Percentage  of  Total  Red  Agents  Detected 

Figure  12  represents  the  distributions  of  the  percentage  of  red  agents  detected 
when  the  ground  radars,  along  with  the  one  or  more  UAVs,  are  implemented  in  the 
improved  configuration.  An  average  of  81.24%  of  red  agents  ±  1%  was  detected  at  a  95% 
Cl,  which  is  less  than  the  required  MOE  1  threshold  of  90%  of  red  agents  detected. 

The  implementation  of  UAVs  in  the  operation  for  the  GCF  shows  an  increase  of 
detection  by  almost  49%.  The  important  factors  that  led  to  this  increase  are  discussed  in 
later  sections. 
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Figure  12.  Distribution  and  Summary  Statistics  for 
Improved  Configuration  (MOE  1). 


2.  MOE  2:  Time  It  Takes  to  Detect  40%  of  the  Red  Agents 

Figure  13  shows  the  total  time  it  takes  to  detect  40%  of  the  red  agents  in  the 
operation.  Adding  the  UAVs  will  benefit  the  GCF  by  speeding  up  the  process  of 
detecting  the  red  agents,  allowing  GCF  to  determine  the  center  of  activity  to  pursue.  This 
will  enhance  the  probability  of  successfully  securing  the  border. 
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Figure  13.  Distribution  and  Summary  Statistics  for 
Improved  Configuration  (MOE  2). 


The  data  distribution  for  MOE2  shows  a  non-symmetrical  distribution  (Figure 
13).  An  average  time  of  6.8  hours  to  detect  40%  of  the  red  agents  has  been  achieved.  This 
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average  mean  value  lands  around  the  65th  percentile  in  the  distribution,  which  means 
35%  of  the  data  can  detect  the  red  agents  in  shorter  periods. 

The  introduction  of  the  UAVs,  along  with  the  available  ground  radars,  has 
apparently  improved  the  speed  of  the  detection  in  comparison  with  the  baseline 
configuration,  but  it  still  did  not  meet  the  threshold  value  of  the  MOE  2  (3.55  hours). 

3.  Conclusion  for  the  Improved  Configuration 

The  introduction  of  the  UAVs  with  their  current  capabilities,  along  with  the 
available  ground  radars,  has  significantly  improved  the  percentage  of  red  agent  detection 
and  the  time  to  detect  them,  but  it  did  not  achieve  the  required  threshold  for  either  of  the 
MOEs.  This  means  that  there  should  be  further  investigation  into  what  improvement  in 
UAV  capabilities  (United  States  Anny  2010)  will  increase  the  percentage  and  speed  of 
detection.  This  investigation  will  aid  the  GCF’s  decision  in  what  specific  capabilities  they 
should  invest. 

D.  EXAMINATION  OF  THE  UAV’S  SIGNIFICANT  CAPABILITIES 

We  claim  that  a  specific  number  of  UAVs  with  certain  capabilities  will  achieve 
the  required  threshold  for  both  MOEs  for  the  GCF  to  accomplish  mission  success.  The 
next  section  examines  the  possible  combination  of  UAV  capabilities  that  could  succeed. 

Introducing  the  UAV  technology  to  support  the  detection  for  the  GCF  has  shown 
apparent  improvement,  but  it  did  not  reach  the  specified  operational  goals.  Thus,  the  next 
section  considers  the  method  of  determining  what  particular  UAV  characteristics  are 
most  valuable.  First,  we  plot  the  correlation  between  the  two  selected  MOEs  to  better 
observe  whether  one  or  more  design  points  can  satisfy  both  MOEs.  Second,  we  examine 
the  significant  factors  that  have  the  greatest  impact  on  the  UAV  through  regression 
analysis.  Third,  we  use  a  partition  tree  analysis  to  identify  the  design  point  options  that 
can  meet  both  MOEs.  Finally,  we  perform  a  relative  cost  and  benefit  analysis  to  discover 
the  desirable  design  point  option. 
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1. 


Correlation  between  MOEs 


The  two  MOEs  are  correlated  when  shown  graphically  (Figure  14).  The  graphical 
presentation  helps  to  identify  possible  design  points  that  could  satisfy  the  operational 
requirements  for  the  GCF.  Figure  14  shows  a  scatter  plot  between  the  two  MOEs  with 
their  threshold  levels  and  possible  DPs  that  can  achieve  mission  success.  As  shown  in 
Figure  14,  multiple  DPs  fall  within  the  operational  goal  target,  represented  by  the  shaded 
area. 


1  3  5  7  9  11  13  15  17 

MOE2  Time  to  Detect  40%  of  Red  Agents 


Figure  14.  Correlation  between  MOEs. 


2.  Significant  Factors  Selection 

Identifying  the  most  significant  factors  of  the  UAV  will  help  the  GCF  identify  the 
UAV  best  suited  for  the  required  detection  mission.  Examining  the  factors  will  help  in 
focusing  on  only  those  factors  related  to  improving  the  detection  capability  of  the  UAV. 
We  use  regression  analysis  to  discover  the  significant  factors  for  the  UAV. 
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a.  MOE 1  Significant  Factors 

A  10-factor  stepwise  linear  regression  model  results  in  an  adjusted  R  square  value 
of  0.68,  which  indicates  that  the  regression  model  developed  is  adequate  to  predict  the 
behavior  of  MOE  1 .  The  stepwise  regression  analysis  examines  how  much  each  of  those 
factors  can  add  to  the  description  of  the  MOE  1,  which  will  identify  the  important  factors. 
This  first  step  screens  all  significant  factors  for  MOE  1.  We  do  not  consider  interaction 
tenns  in  this  regression  model.  Figure  15  lists  all  the  factors  with  the  largest  effect  on 
MOE  1. 


Sorted  Parameter  Estimates 


Term 

Estimate 

Std  Error 

Red  Stealth 

-0.00847 

0.000602 

UAVMaxDetRng 

5.61 29e-5 

0.000004 

NumUAVs 

0.0327168 

0.003152 

UAVAperture 

0.0016761 

0.000201 

UAVEndurance.hrs. 

0.0078887 

0.001703 

U  AVA  Ititud  e_Def  a  u  It 

-3.593e-6 

2.405e-6 

UAVMaxTimeBtwDet 

-0.000911 

0.000798 

UAVSIewRate 

-2.31 7e-5 

4.01 2e-5 

U  A  VSpeed_Def  a  u  It 

-2.948e-5 

0.0001 

U  A  VRef  u  elTime.  m  i  n . 

3.1 1 34e-5 

0.000399 

Prob>|t| 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

0.1365 

0.2545 

0.5641 

0.7690 

0.9379 


Figure  15.  Significant  Factors  for  MOE  1. 


The  significance  level  (a)  was  selected  to  be  0.05,  which  is  the  limit  to  reject  or 
fail  to  reject  a  factor.  The  t  Ratio  value  is  a  test  statistic  for  the  hypothesis  test  to  evaluate 
whether  the  estimate  for  that  factor  is  significant.  Patrick  Runkel  (2016)  in  his  blog 
section  T  &  P  explains  that  “t-value  measures  the  size  of  the  difference  relative  to  the 
variation  in  the  sample  data.  It  is  the  calculated  difference  represented  in  units  of  standard 
error.”  The  greater  the  t-value,  the  greater  the  evidence  to  reject  the  null  hypothesis  to 
which  the  associated  population  parameter  is  plausibly  equal.  The  null  hypothesis  is  that 
the  true  value  of  the  coefficient  related  to  the  factor  is  0;  indicating  that  it  has  no 
influence  on  the  MOE. 


36 


The  estimate  value  in  the  second  column  is  the  amount  of  change  to  MOE  1  per 
one  unit  change  in  the  factor-the  estimated  coefficient.  For  example,  the  red  stealth 
estimate  has  a  negative  value  of  (-0.00847)  because  as  red  stealth  increases  one  unit,  the 
percentage  of  red  agents  detected  decreases  by  this  value.  Another  case  is  the  detection 
range  factor.  It  can  be  explained  as:  for  every  1 -meter  increase  in  detection  range,  MOE  1 
increases  5.6e-5  percent. 

A  second  order  stepwise  regression  analysis  was  performed  to  find  significant 
factors  that  interact  with  other  factors.  It  results  in  an  adjusted  R  square  of  0.73,  which 
indicates  that  the  regression  model  developed  is  adequate  to  predict  the  behavior  of  MOE 
1.  Figure  16  shows  the  significant  factors  of  the  second  order  with  their  associated  values. 
Five  factors  were  significant  as  shown  in  the  Prediction  Profiler:  Red  Stealth,  UAV 
Maximum  detection  range,  Number  of  UAVs,  UAV  Aperture,  and  the  UAV  Endurance. 


Sorted  Parameter  Estimates 

Term  Estimate  Std  Error  t 

RedStealth  -0.008464  0.000551  - 

UAVMaxDetRng  5.6055e-5  3.674e-6 

NumUAVs  0.0327168  0.00289 

UAVAperture  0.0016795  0.000184 

(NumUAVs-2.5)*(NumUAVs-2.5)  -0.018226  0.003231 

UAVEndurance.hrs.  0.0080077  0.001558 

(UAVAperture-90.0308)*(RedStealth-75.0308)  0.0001045  3.25e-5 

(NumUAVs-2.5)*(RedStealth-75.0308)  0.0013708  0.000493 

Effect  Details 

»  Prediction  Profiler 


Prob>|t| 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

0.0015’ 

0.0058* 


RedStealth  NumUAVs  UAVMaxDetRng  UAVAperture  UAVEndurance.hrs 


Figure  16.  Second  Order  Stepwise  Linear  Regression  for  MOE  1. 


The  Prediction  Profiler  shows  an  interesting  result:  the  Number  of  UAVs  factor 

(as  a  main  factor)  shows  that  increasing  the  number  of  UAVs  beyond  three  provides 

marginal  improvement  to  MOE  1.  This  could  be  because  three  UAV  sets  saturate  the 
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operational  area  and  we  do  not  need  an  extra  UAV  set.  This  factor  requires  further  study 
on  route  development  and  sensor  capabilities,  but  that  is  beyond  the  scope  of  this  thesis. 

Figure  17  shows  a  matrix  of  interaction  between  the  factors.  A  non-parallel  line 
gives  an  indication  of  possible  interaction  between  the  factors  and  their  effect  on  that 
MOE.  For  example,  the  highlighted  interaction  between  the  UAV  Aperture  and  Red 
Stealth  factors  can  be  explained  as  follows:  when  Red  Stealth  is  low  (Value  65), 
changing  the  UAV  Aperture  (from  60  to  130  degrees)  has  minimal  effect  on  MOE  1.  By 
contrast,  when  Red  Stealth  is  high  (Value  85),  increasing  UAV  Aperture  (from  60  to  130 
degrees)  has  larger  impact  on  MOE  1 . 


Interaction  Profiles 
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Figure  17.  Factor  Interaction  MOE  1. 


b.  MOE  2  Significant  Factors 

Another  10-factor  linear  regression  model  was  conducted  for  MOE  2  that  results 
in  an  adjusted  R  square  of  0.80,  which  indicates  that  the  regression  model  is  adequate  to 
predict  the  behavior  of  MOE  2.  With  the  same  significance  level  selected  for  MOE  1  of 
0.05,  Figure  18  shows,  in  order  of  importance,  the  significant  factors  that  have  an  impact 
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on  MOE  2.  As  seen  in  Figure  18,  five  factors  have  an  impact  on  MOE  2.  The  Estimate 
value  for  the  number  of  UAVs  factor  shows  a  negative  value,  which  means  that  as  the 
number  of  UAVs  increases,  the  time  it  takes  to  detect  40%  of  the  red  agents  decreases. 
Similarly,  in  the  UAV  endurance  factor,  the  greater  the  UAV  endurance  means  the  less 
time  we  need  to  detect  40%  of  the  red  agents.  This  is  because  the  UAV  will  not  waste 
time  when  refueling  on  the  ground  and  will  spend  more  time  in  the  air.  The  Red  Stealth 
factor,  on  the  other  hand,  shows  a  positive  value,  which  reflects  the  following:  as  the  red 
stealth  level  increases,  the  time  it  takes  to  detect  the  red  agents  increases  as  well. 


Sorted  Parameter  Estimates 


Term 

NumUAVs 

UAVMaxDetRng 

Red  Stealth 

UAVAperture 

UAVEndurance.hrs. 

UAVAItitude_Defau  It 

UAVMaxTimeBtwDet 

UAVSIewRate 

U  A  VRef  u  elTi  me.  m  i  n . 

U  A  VSpeed_Def  a  u  It 


Estimate 

-1.858138 

-0.001803 

0.1952902 

-0.05267 

-0.306637 

0.0001244 

0.0287277 

0.0007187 

-0.004073 

0.0003005 


Std  Error 

0.084219 

0.000107 

0.016078 

0.005364 

0.045507 

6.427e-5 

0.021321 

0.001072 

0.010664 

0.002679 


t  Ratio 


Prob>|t| 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

0.0541 

0.1791 

0.5032 

0.7028 

0.9108 


Figure  18.  Significant  Factors  for  MOE  2. 


Performing  a  second  order  stepwise  regression  model  has  resulted  in  a  higher 
adjusted  R  square  value  of  0.82  and  an  additional  factor  that  might  have  an  impact  on 
MOE  2.  Figure  19  illustrates  the  significant  factors  of  the  second  order  with  their 
associated  values. 
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J  Sorted  Parameter  Estimates 

Term  Estimate  Std  Error  t  Ratio 


NumUAVs 

UAVMaxDetRng 

RedStealth 

UAVAperture 

UAVEndurance.hrs. 

(UAVSpeed_Default-140.062)*(UAVApertu  re-90, 0308) 
(NumUAVs-2.5)’(NumUAVs-2.5) 
(UAVAperture-90.0308)*(RedStealth-75.0308) 
UAVSpeed_Default 


-1.858138  0,079822  -23  28 
-0.001801  0.000101  -17.74 
0.1950923  0.015232  12.81 
-0.052836  0.005084  -1039 
-0.310894  0.043021  -7.23 

-0.000521  0.000143  -3.64 

0.3197308  0.089243  3.58 

-0.002841  0.000901  -3.15 

0.0001853  0.002538  0.07 


Prob>  |t| 

<.0001* 

<.0001* 

<■0001* 

<.0001* 

<.0001* 

0.0003* 

0.0004* 

0.0018* 

0.9419 


Effect  Details 


A  »  Prediction  Profiler 


1 11!  6.431884  ^ 
c  |  5  [6.15052.  I 
|  o  6.71325] 

2  I-  O  J 


H  HM  Mfflrfl  tr 


75.031  2.5  2  500.1  90.031  4.5077  140.06 

RedStealth  NumUAVs  UAVMaxDetRng  UAVAperture  UAVEndurance.hrs.  UAVSpeed_Default 


Figure  19.  Second  Order  Stepwise  Linear  Regression  MOE  2. 


The  UAV  Speed  as  a  main  factor  has  no  effect  on  MOE  2  as  shown  in  the 
Prediction  Profiler  (almost  flat  curve  line),  but  it  will  have  an  effect  when  interacting 
with  other  factors  (Figure  20).  The  interaction  matrix  for  the  MOE  2  significant  factors 
shows  an  interaction  between  the  UAV  Speed  factor  and  the  UAV  Aperture  factor  over 
MOE  2.  When  the  Speed  of  the  UAV  is  high  (value  =  200),  increasing  the  UAV  Aperture 
(from  60  to  130  degrees)  has  a  larger  impact  on  MOE  2. 
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Figure  20.  Factor  Interaction  MOE  2. 


As  a  result  of  this  interesting  finding  about  the  interaction  between  factors,  we 
decide  to  use  the  UAV  Speed  as  a  significant  factor.  Therefore,  a  total  of  six  factors  were 
significant  for  MOE  2:  Red  Stealth,  UAV  Maximum  detection  range,  Number  of  UAVs, 
UAV  Aperture,  UAV  Endurance,  and  UAV  Speed. 

c.  Major  Significant  Factors 

While  MOE  2  has  the  UAV  Speed  as  an  additional  significant  factor  that 
influences  it,  the  two  MOEs  have  five  of  the  factors  in  common.  Thus,  a  total  of  six 
factors  have  a  significant  effect  on  the  UAV’s  ability  to  improve  GCF’s  capability  to 
detect  and  respond  to  infiltrators. 
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3. 


Design  Points  that  Meet  Both  MOEs 


We  plotted  the  correlation  between  the  MOEs  in  Figure  14  and  highlighted  the 
intersection  area  of  design  points  that  meet  the  threshold  level  of  both  MOEs.  A  total  of  a 
32  design  points  met  the  criteria  and  are  used  for  further  analysis  (Table  5). 


Table  5.  Design  Points  that  Meet  MOE  1  and  MOE  2. 


No# 

DP 

NumUAV 

UAVSpeed 

UAVAItit 

UAVMaxDetecRange 

TimeB/wDetec 

Endurance 

RefulTime 

SlewRate 

Aperture 

RedStealth 

1 

79 

2 

149 

4641 

3344 

19 

4 

37 

201 

116 

68 

2 

106 

2 

118 

7844 

3953 

8 

5 

42 

243 

114 

69 

3 

110 

2 

134 

6906 

3016 

10 

5 

45 

304 

103 

66 

4 

140 

3 

142 

3078 

3906 

13 

2 

43 

154 

101 

65 

5 

141 

3 

198 

5578 

3063 

19 

2 

32 

173 

109 

71 

6 

144 

3 

149 

4641 

3344 

19 

4 

37 

201 

116 

68 

7 

150 

3 

179 

4016 

2219 

6 

8 

41 

65 

102 

71 

8 

159 

3 

164 

7688 

2031 

16 

7 

60 

163 

92 

66 

9 

163 

3 

114 

7766 

2922 

15 

7 

37 

121 

91 

66 

10 

171 

3 

118 

7844 

3953 

8 

5 

42 

243 

114 

69 

11 

172 

3 

84 

4563 

2547 

9 

8 

52 

83 

98 

72 

12 

175 

3 

134 

6906 

3016 

10 

5 

45 

304 

103 

66 

13 

176 

3 

106 

5266 

2734 

7 

6 

31 

346 

120 

75 

14 

201 

4 

172 

3938 

2594 

5 

3 

40 

327 

85 

68 

15 

205 

4 

142 

3078 

3906 

13 

2 

43 

154 

101 

65 

16 

206 

4 

198 

5578 

3063 

19 

2 

32 

173 

109 

71 

17 

209 

4 

149 

4641 

3344 

19 

4 

37 

201 

116 

68 

18 

211 

4 

183 

5188 

2172 

7 

6 

60 

294 

118 

77 

19 

212 

4 

148 

7297 

1797 

10 

5 

44 

308 

108 

72 

20 

215 

4 

179 

4016 

2219 

6 

8 

41 

65 

102 

71 

21 

221 

4 

181 

3469 

3250 

17 

5 

57 

158 

117 

80 

22 

224 

4 

164 

7688 

2031 

16 

7 

60 

163 

92 

66 

23 

226 

4 

185 

6281 

3813 

18 

6 

49 

332 

100 

78 

24 

227 

4 

140 

5500 

2500 

13 

5 

45 

210 

90 

75 

25 

228 

4 

114 

7766 

2922 

15 

7 

37 

121 

91 

66 

26 

229 

4 

86 

4406 

3672 

14 

6 

53 

196 

75 

71 

27 

233 

4 

136 

3391 

3297 

13 

7 

40 

318 

68 

67 

28 

236 

4 

118 

7844 

3953 

8 

5 

42 

243 

114 

69 

29 

237 

4 

84 

4563 

2547 

9 

8 

52 

83 

98 

72 

30 

240 

4 

134 

6906 

3016 

10 

5 

45 

304 

103 

66 

31 

241 

4 

106 

5266 

2734 

7 

6 

31 

346 

120 

75 

32 

246 

4 

112 

6047 

2313 

16 

3 

33 

351 

96 

74 

4.  Partition  Tree  Analysis 

We  use  a  partition  tree  analysis  to  find  the  most  consistent  or  predictable  way  to 
satisfy  both  MOEs.  This  method  enables  us  to  explain  the  predicted  value  through  a 
multiple  combination  of  factors.  The  partition  tree  moves  from  top  to  bottom,  searching 
for  possible  splits  between  factors,  and  it  indicates  the  best  split,  based  on  greater  values. 
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The  graphical  tree  structure  provides  an  easy  way  to  understand  and  interpret  data.  It 
allows  for  discovering  a  correlation  between  factors  in  the  data.  Readers  should  note  that 
every  design  point  (DP)  is  a  mixture  of  different  factor  levels  (refer  to  Table  5),  and  each 
DP  is  one  UAV  possible  option. 

a.  Partition  Tree  Analysis  for  Both  MOEs 

To  improve  the  detection  capability  for  the  GCF  and  increase  border  security,  we 
aim  to  meet  both  MOE  thresholds.  As  discussed  in  the  Design  Point  section,  32  DPs  out 
of  260  satisfied  the  criteria  of  both  MOEs. 

The  following  description  is  a  simplified  version  of  the  tree  analysis  on  which 
factors  contribute  most  to  satisfying  both  MOEs.  For  more  detailed  analysis  with  exact 
values,  please  refer  to  Appendix  D. 

The  partition  tree  in  Figure  21  represents  the  breakdown  of  the  overall  (260) 
design  points.  The  top  box  contains  the  DPs  that  met  both  MOEs  (32  DPs)  marked  with 
“YES.”  The  first  split  appears  on  the  red  stealth  level.  Splitting  the  tree  with  the  Red 
Stealth  factor  can  assist  the  GCF  in  determining  the  ideal  UAV  in  extreme  cases  for 
detecting  red  agents  when  they  are  most  and  least  capable  of  being  detected. 


UAV>=3 
Count  48 

"YES"  22 


UAV<3 

Count 

48 

"YES" 

3 

UAV>=4 

Count 

41 

"YES" 

6 

UAV<4 

Count 

123 

"YES" 

1 

Figure  2 1 .  Partition  Tree  for  MOEs. 
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As  the  split  on  this  level  indicates,  the  Low  Red  Stealth  factor  has  96  DPs  among 
the  260,  but  only  25  of  them  can  achieve  both  MOEs.  The  High  Red  Stealth  factor  has 
164  DPs  out  of  260,  but  only  seven  can  meet  the  MOEs’  criteria. 

We  further  split  the  tree  to  see  how  the  UAV  factors  contribute  to  the  success  in 
the  low  and  high  red  stealth  cases.  The  next  split  under  the  low  stealth  cases  is  based  on 
the  number  of  UAVs,  which  splits  evenly:  48  DPs  on  both  sides.  The  “YES”  criteria  was 
achieved  22  times  when  the  number  of  UAVs  was  three  or  more,  while  it  was  achieved 
only  three  times  when  UAVs  were  fewer  than  three. 

The  high  red  stealth  path  shows  seven  possible  DPs  that  can  achieve  both  MOEs. 
Only  one  DP  can  possibly  meet  both  MOEs  with  less  than  four  UAVs.  The  other  six  DPs 
require  four  UAVs. 

We  can  further  split  the  tree  to  better  analyze  the  data  and  better  understand  which 
factor  will  have  an  effect  on  the  achievement  of  the  MOE  thresholds  (see  Appendix  D). 

b.  Partition  Tree  Outcomes 

Solutions  using  more  UAVs  tend  to  give  better  results  or  more  options  to  achieve 
the  MOEs’  threshold,  but  the  marginal  improvement  after  the  third  UAV  set  does  not 
appear  to  justify  the  cost.  In  the  next  section,  however,  we  examine  the  benefit  of  an 
additional  UAV  in  comparison  to  relative  costs. 

There  are  only  a  few  design  point  options  to  examine  when  the  enemy  is  highly 
stealthy.  Seven  DPs  have  achieved  the  threshold  for  both  MOEs  in  the  high  red  stealth 
configuration.  One  of  those  DPs  can  meet  both  MOE  thresholds  with  less  than  four 
UAVs. 

In  the  other  scenario  when  the  enemy  is  less  stealthy,  25  design  point  options 
achieved  the  threshold  for  both  MOEs.  Using  at  least  three  UAV  sets  appears  to  provide 
more  options  to  detect  at  least  90%  of  the  red  agents  in  3.55  hours  or  less.  The  partition 
tree  in  Figure  21  shows  three  design  point  options  that  meet  both  MOEs’  thresholds  using 
fewer  than  three  UAV  sets.  Further  experimentation  follows  in  this  thesis  to  show  which 
DP  option  is  the  most  cost-effective  combination. 
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5. 


UAV  Capability  and  Relative  Cost  Analysis  Tradeoff 


Following  Sangbum  Kim’s  (2017)  approach  to  developing  a  relative  cost  analysis 
will  help  identify  the  best  UAV  capabilities  at  the  most  effective  cost.  Assigning  each 
UAV  capability  with  a  value  assists  in  computing  each  design  point’s  final  relative  cost. 
The  optimum  design  point  should  have  a  high  MOE  value  and  lowest  relative  cost. 
However,  a  UAV’s  component  cost  breakdown  is  unavailable  and  hard  to  obtain  within 
the  time  frame  of  this  thesis.  Therefore,  we  create  a  relative  cost  scale  for  each  UAV 
capability  in  terms  of  the  user  priorities  to  develop  an  estimate  cost  for  each  DP.  A 
comparison  between  the  DP  final  relative  cost  and  its  MOE  value  will  aid  in  defining  the 
option  we  desire  to  pursue. 

To  calculate  the  expected  cost  for  each  design  option,  we  need  a  reference  UAV 
model  upon  which  to  base  our  calculation.  As  mentioned  in  Chapter  III,  a  UAV  from 
Group  1  to  Group  3  was  used  to  set  the  factor  ranges  in  this  study.  The  Raven  RQ-11 
UAV  (Figure  22)  used  by  the  U.S.  military  (Kasper  2014)  falls  into  this  category  and  is 
used  as  a  cost  reference  for  this  study. 


Figure  22.  Raven  RQ-1 1  UAV.  Source:  AeroVironment.inc  (2017). 


The  estimated  cost  of  the  Raven  RQ-1 1  is  $300,000  USD  for  a  set  of  three  UAVs 
with  the  two  ground  control  stations  and  its  supporting  equipment  (Kasper  2014).  The 
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price  of  one  UAV  vehicle  is  estimated  at  $34,000.  The  author  has  derived  a  price 
breakdown  of  the  Raven  RQ-1 1  system  (Table  6). 


Table  6.  Author-derived  Raven  RQ-1 1  Cost  Breakdown. 


Raven  RQ- 1 1  Cost 

$  300,000 

Vehicle  Cost 

$  34,000 

Vehicle  Cost  x  3  (Three  vehicles) 

$  102,000 

GCS  (x  2)  +  Supporting  equipment 

$  198,000 

a.  Factor  Relative  Cost  Scale 

The  cost  of  each  design  point  is  a  product  of  the  UAV  capability  factor  cost. 
Therefore,  we  need  to  compute  the  relative  cost  of  the  eight  UAV  capability  factors.  The 
Analytic  Hierarchy  Process  (AHP)  can  aid  in  calculating  UAV  factor  weights  for  this 
study. 

The  AHP  process  runs  a  pairwise  comparison  between  the  factors  based  on  which 
factor  has  more  importance  in  terms  of  UAV  performance  (Table  7).  The  more  important 
the  factor  in  regards  to  the  UAV  performance,  the  higher  the  relative  cost  it  will  have. 


Table  7.  UAV  Eight-Capability  Factor  Pairwise  Comparison. 


UAVSpeed 

UAVAItit 

UAVMaxDetecRange 

TimeBwDetec 

Endurance 

RefuelTime 

SlewRate 

Aperture 

UAVSpeed 

1 

1/3 

1/4 

3 

1/5 

2 

1/2 

1/6 

UAVAItit 

3 

1 

1/2 

5 

1/3 

4 

2 

1/4 

UAVMaxDetecRa  nge 

4 

2 

1 

6 

1/2 

5 

3 

1/3 

TimeBwDetec 

1/3 

1/5 

1/6 

1 

1/7 

1/2 

1/4 

1/8 

Endurance 

5 

3 

2 

7 

1 

6 

4 

1/2 

RefuelTime 

1/2 

1/4 

1/5 

2 

1/6 

1 

1/3 

1/7 

SlewRate 

2 

1/2 

1/3 

4 

1/4 

3 

1 

1/5 

Aperture 

6 

4 

3 

8 

2 

7 

5 

1 

Normally,  a  user  compares  a  single  factor  against  other  factors  and  assigns  a  score 
depending  on  how  important  it  is  (from  one  to  nine,  where  one  is  equally  preferred  and 
nine  is  extremely  preferred)  (Bodin  and  Gass  2003).  For  this  study,  however,  the  factor’s 
values  for  importance  are  informed  by  the  regression  analysis  results  in  the  data 
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correlation  section.  The  Estimate  column  in  the  regression  analysis  shows  how  much 
change  in  the  MOE  can  be  caused  by  the  one-unit  change  in  the  factor.  For  example,  in 
the  MOE  1  regression  analysis,  the  UAV  aperture  factor  has  a  greater  effect  on  the 
percentage  of  detected  red  agents  over  UAV  endurance,  and  hence,  the  UAV  aperture  is 
preferred  over  the  UAV  endurance  factor  and  receives  a  higher  value. 

Data  in  Table  7  represent  the  importance  value  when  comparing  the  row  factor  to 
the  column  factor.  An  integer  ( 1  to  9)  means  the  factor  in  the  row  is  more  preferred  to  the 
factor  in  the  column  by  the  integer  amount.  A  fraction  of  the  integer  indicates  that  the 
factor  in  the  row  is  less  preferred  than  the  factor  in  the  column. 

The  next  step  is  to  normalize  the  data  in  the  matrix  shown  in  Table  7  to  obtain  the 
factors’  weight  scores.  First,  we  sum  each  of  the  pairwise  comparison  columns;  then,  we 
divide  each  entry  cell  in  the  matrix  by  its  column  sum  value  (Bodin  &  Gass  2003).  The 
average  value  in  each  row  of  the  normalized  matrix  serves  as  the  weight  value  for  the 
UAV  capability  factor.  Table  8  presents  the  score  results,  which  are  highlighted  in 
orange. 


Table  8.  UAV  Eight  Factors’  Weight  Scores. 


Normalizing  and  Comparisons 


UAV 

Speed 

UAV 

Altit 

UAV 

Max 

Detec 

Range 

Time 

Bw 

Detec 

Endur¬ 

ance 

Refuel 

Time 

Slew 

Rate 

Aper¬ 

ture 

Score 

UAVSpeed 

0.05 

0.03 

0.03 

0.08 

0.04 

0.07 

0.03 

0.06 

0.05 

UAVAltit 

0.14 

0.09 

0.07 

0.14 

0.07 

0.14 

0.12 

0.09 

0.11 

UAVMaxDetecRange 

0.18 

0.18 

0.13 

0.17 

0.11 

0.18 

0.19 

0.12 

0.16 

TimeBwDetec 

0.02 

0.02 

0.02 

0.03 

0.03 

0.02 

0.02 

0.05 

0.02 

Endurance 

0.23 

0.27 

0.27 

0.19 

0.22 

0.21 

0.25 

0.18 

0.23 

RefuelTime 

0.02 

0.02 

0.03 

0.06 

0.04 

0.04 

0.02 

0.05 

0.03 

SlewRate 

0.09 

0.04 

0.04 

0.11 

0.05 

0.11 

0.06 

0.07 

0.07 

Aperture 

0.27 

0.35 

0.40 

0.22 

0.44 

0.25 

0.31 

0.37 

0.33 

Lawrence  Bodin  and  Saul  Gass  explain  the  10%  rule,  which  refers  to  the 
consistency  level  that  AHP  users  have  to  maintain.  It  reflects  that  users  were  consistent  in 
the  preference  rating  in  the  pairwise  matrix.  Our  preference  rating  in  Table  7  has  resulted 
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in  4.1%,  which  is  a  satisfactory  ratio  of  consistency  (Bodin  &  Gass  2003).  Refer  to 
Appendix  E  for  the  full  calculations. 

b.  Design  Points  Relative  Cost  Calculation 

Table  9  illustrate  the  process  the  author  used  to  calculate  the  relative  cost  for  each 
design  point.  First,  we  break  down  the  cost  of  one  vehicle  of  the  Raven  RQ-11  UAV 
based  on  the  calculated  UAV  capability  weights  in  Table  8.  The  second  row  (cost  per 
factor)  shows  the  cost  of  each  capability  in  regards  to  the  relative  weights  of  the  factor. 
Next,  we  compare  each  Raven  RQ-11  capability  to  the  DP  capability  and  compute  the 
percentage  difference.  The  resulting  percentage  is  then  multiplied  by  the  Raven  RQ-11 
capability  relative  cost  (cost  per  factor  row)  to  calculate  the  required  DP  relative  cost. 
Table  9  shows  (DP  1)  relative  cost  calculation,  following  this  process,  has  resulted  in  a 
total  relative  cost  for  DP  1  equals  $65,440. 

There  are  some  limitations  to  the  relative  cost  calculation  used  in  this  study  based 
on  the  assumptions  made  by  the  author.  Each  design  point  is  an  enhancement  or  a 
decrement  from  the  reference  Raven  RQ-1 1  capabilities.  The  cost  for  the  enhancement  is 
proportional  and  linear  to  the  importance  of  the  characteristic.  We  alert  the  readers  that 
this  is  an  estimation  for  the  DP  cost  and  not  a  precise  cost  value. 


Table  9.  Design  Point  Relative  Cost  Calculation. 


UAV 

Speed 

UAV 

Altit 

UAV 

Max 

Detec 

Time 

Bw 

Detec 

Endur¬ 

ance 

Refuel 

Time 

Slew 

Rate 

Aper¬ 

ture 

Sum 

$  34,000.00 

0.050 

0.108 

0.157 

0.024 

0.227 

0.034 

0.073 

0.327 

i 

cost  per  factor 

$1,693.06 

$3,660.52 

$5,333.03 

$821.63 

$7,729.54 

$1,156.52 

$2,495.72 

$11,109.97 

$34,000.00 

Raven  RQ-11  capability 

80 

3000 

1000 

20 

1 

60 

60 

60 

DP  capability 

166 

3234 

2078 

10 

2 

53 

299 

89 

%  Change 

1.08 

0.08 

1.08 

0.50 

1.00 

0.12 

3.98 

0.48 

%  Change  cost 

$1,820.04 

$  285.52 

$5,749.01 

$410.82 

$7,729.54 

$  134.93 

$9,941.30 

$  5,369.82 

$31,440.97 

cost  1  UAV 

$65,440.97 

The  resulting  relative  cost  for  one  UAV  in  Table  9  is  multiplied  by  three  (for  a  set 


of  three  UAVs)  and  then  added  to  the  cost  of  the  two  ground  control  stations  and 
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supporting  equipment.  A  similar  methodology  is  used  for  all  260  DPs  to  calculate  each 
DP’s  total  relative  cost  to  the  system  (see  Appendix  F). 

c.  Low  Red  Stealth  Relative  Cost  Tradeoff 

All  MOEs’  values  for  low  red  stealth  DPs  will  be  plotted  against  the  total  system 
relative  cost.  This  can  better  illustrate  the  best  DP  that  maximizes  the  MOE  value  for  the 
lowest  relative  cost  value  in  both  case  scenarios. 

(1)  MOE  1  vs.  Relative  Cost  (Low  Stealth) 

Figure  23  presents  each  MOE  1  DP  with  its  total  system  relative  cost  value.  The 
dashed  black  line  is  the  MOE  #1  threshold.  DPs  in  the  red  circle  represent  the  DPs  that 
achieve  the  threshold  for  both  MOEs.  The  orange  line  represents  the  relative  efficiency 
frontier  line  that  connects  the  DPs  with  the  highest  MOE  #1  value  at  their  relative  total 
cost  value. 


0.6 


0.5 


0.4  1 - 

$200,000.00 


$700,000.00 


$1,200,000.00  $1,700,000.00 

Total  cost 


A  MOE  1 
•  Best  MOE  1 


$2,200,000.00 


$2,700,000.00 


Figure  23.  Relative  Efficiency  Frontier  for  MOE  1  (Low  Red  Stealth). 
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As  shown  in  Figure  23,  DP  79  meets  both  MOEs’  thresholds  with  the  lowest 
relative  total  system  cost  of  $935,222.  Other  DPs  meet  both  MOEs  thresholds,  but  with  a 
more  expensive  relative  total  system  cost. 

The  cheapest  option  that  reaches  the  threshold  of  MOE  1  only  is  DP  14  (90.35%). 
It  has  a  relative  total  system  cost  that  is  almost  half  the  DP  79  ($467,61 1.49),  but  DP  14’s 
time  to  detect  40%  of  the  red  agents  is  higher  than  the  MOE  2  threshold  of  5.93  hours. 

(2)  MOE  2  vs.  Relative  Cost  (Low  Stealth) 

Figure  24  presents  each  MOE  2  DP  with  its  relative  system  cost.  The  dashed 
black  line  represents  the  MOE  2  threshold  of  3.55  hours.  Similarly,  DPs  in  the  red  circle 
represent  the  DPs  that  achieve  the  threshold  for  both  MOEs.  The  orange  line  represents 
the  relative  efficiency  frontier  line  that  connects  the  DPs  with  the  lowest  MOE  #2  value. 


Figure  24.  Relative  Efficiency  Frontier  for  MOE  2  (Low  Red  Stealth). 
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As  shown  in  Figure  24,  DP  79  has  the  lowest  relative  cost  and  satisfies  both 
MOEs  with  an  MOE  2  value  of  3.28  hours.  No  other  DP  can  reach  the  MOE  2  threshold 
at  cheaper  relative  cost.  Other  DPs  fall  into  the  relative  efficiency  frontier  curve  and 
could  deliver  less  time  than  DP  79,  but  they  will  relatively  cost  more.  For  example,  DP 
209  can  expend  the  least  time  to  detect  40%  of  the  red  agents  (1 .77  hours)  but  will  have  a 
relative  cost  of  exactly  double  the  price  $1,870,445.94  as  that  of  DP  79. 

(3)  Low  Stealth  Design  Points  Relative  Cost 

Table  10  presents  the  low  red  stealth  design  points  with  their  relative  cost.  These 
relative  costs  serve  only  as  a  guide  for  the  GCF  because  actual  UAV  component  cost 
breakdowns  were  hard  to  obtain  during  the  timeframe  of  this  study.  Actual  costs  would 
have  to  be  considered  for  comparison  if  GCF  needs  to  purchase  a  UAV  with  greater 
capability  to  fulfill  its  needs. 
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Table  10.  Low  Red  Stealth  DP  Relative  Cost. 


DP 

Num 

UAV 

UAV 

Speed 

UAV 

Altit 

UAV 

Max 

Detec 

Range 

Time 

Bw 

Detec 

Endur¬ 

ance 

Refuel 

Time 

Slew 

Rate 

Aper¬ 

ture 

Red 

Stealth 

MOE2 

time 

To 

Detect 

40% 

(Hr) 

MOE1 

%Red 

Agent 

Detected 

Relative 

Cost 

79 

2 

149 

4641 

3344 

19 

4 

37 

201 

116 

68 

3.28 

0.96895 

$ 

935,222.97 

106 

2 

118 

7844 

3953 

8 

5 

42 

243 

114 

69 

3.47 

0.95805 

$ 

1,030,993.22 

110 

2 

134 

6906 

3016 

10 

5 

45 

304 

103 

66 

3.37 

0.9676 

$ 

998,338.46 

140 

3 

142 

3078 

3906 

13 

2 

43 

154 

101 

65 

3.29 

0.91215 

$ 

1,230,764.96 

141 

3 

198 

5578 

3063 

19 

2 

32 

173 

109 

71 

3.34 

0.9016 

$ 

1,248,557.99 

144 

3 

149 

4641 

3344 

19 

4 

37 

201 

116 

68 

2.42 

0.9175 

$ 

1,402,834.46 

150 

3 

179 

4016 

2219 

6 

8 

41 

65 

102 

71 

2.89 

0.9027 

$ 

1,555,820.62 

159 

3 

164 

7688 

2031 

16 

7 

60 

163 

92 

66 

2.76 

0.90325 

$ 

1,527,727.19 

163 

3 

114 

7766 

2922 

15 

7 

37 

121 

91 

66 

2.67 

0.9112 

$ 

1,548,796.06 

171 

3 

118 

7844 

3953 

8 

5 

42 

243 

114 

69 

2.57 

0.9117 

$ 

1,546,489.82 

172 

3 

84 

4563 

2547 

9 

8 

52 

83 

98 

72 

3.06 

0.9008 

$ 

1,556,531.10 

175 

3 

134 

6906 

3016 

10 

5 

45 

304 

103 

66 

2.53 

0.9127 

$ 

1,497,507.69 

201 

4 

172 

3938 

2594 

5 

3 

40 

327 

85 

68 

2.29 

0.9216 

$ 

1,725,460.02 

205 

4 

142 

3078 

3906 

13 

2 

43 

154 

101 

65 

2.06 

0.92695 

$ 

1,641,019.94 

206 

4 

198 

5578 

3063 

19 

2 

32 

173 

109 

71 

2.34 

0.9341 

$ 

1,664,743.99 

209 

4 

149 

4641 

3344 

19 

4 

37 

201 

116 

68 

1.77 

0.9449 

$ 

1,870,445.94 

212 

4 

148 

7297 

1797 

10 

5 

44 

308 

108 

72 

2.51 

0.9126 

$ 

1,941,283.65 

215 

4 

179 

4016 

2219 

6 

8 

41 

65 

102 

71 

2.08 

0.94435 

$ 

2,074,427.50 

224 

4 

164 

7688 

2031 

16 

7 

60 

163 

92 

66 

1.94 

0.942 

$ 

2,036,969.58 

228 

4 

114 

7766 

2922 

15 

7 

37 

121 

91 

66 

2.04 

0.93765 

$ 

2,065,061.41 

229 

4 

86 

4406 

3672 

14 

6 

53 

196 

75 

71 

2.61 

0.90785 

$ 

1,962,672.06 

233 

4 

136 

3391 

3297 

13 

7 

40 

318 

68 

67 

2.54 

0.92015 

$ 

2,078,105.80 

236 

4 

118 

7844 

3953 

8 

5 

42 

243 

114 

69 

1.81 

0.94225 

$ 

2,061,986.43 

237 

4 

84 

4563 

2547 

9 

8 

52 

83 

98 

72 

2.15 

0.92205 

$ 

2,075,374.80 

240 

4 

134 

6906 

3016 

10 

5 

45 

304 

103 

66 

1.84 

0.9488 

$ 

1,996,676.91 

Table  10  shows  that  DP  79  is  the  lowest  relative  cost  option  in  addition  to  being 
the  highest  MOE  1  percentage  of  detection.  Therefore,  we  can  conclude  that  DP  79  will 
provide  the  best  relative  cost  efficiency. 

d.  High  Red  Stealth  Relative  Cost  Tradeoff 

If  GCF  decided  that  it  would  go  with  the  high  red  stealth  scenario,  different 
design  point  MOE  values  should  be  plohed  against  the  relative  total  system  cost.  Figure 
2 1  in  the  partition  tree  analysis  section  shows  that  seven  possible  DPs  satisfy  both  MOEs 
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with  a  high  red  stealth  value;  only  one  of  them,  though,  can  achieve  both  MOEs  with  less 
than  four  UAV  sets. 

(1)  MOE  1  vs.  Relative  Cost  (High  Stealth) 

Similarly,  Figure  25  represents  each  MOE  1  DP  with  its  relative  total  system  cost 
for  the  high  red  stealth  case.  DPs  in  red  circles  represent  the  seven  DP  options  that  can 
meet  both  MOEs  for  the  high  red  stealth  scenario.  DP  176  meets  both  MOEs’  thresholds 
with  the  lowest  relative  total  system  cost  of  $1,577,786  for  the  high  red  stealth  scenario. 
DP  176  provides  a  90.2%  detection  of  the  red  agents,  while  DP  241  provides  the  highest 
possible  percentage  of  red  agent  detection  for  the  high  red  stealth  DP  option  of  93.24%. 
This  design  point  option  requires  an  additional  $525k,  however,  when  compared  to  DP 
176. 


Figure  25.  Relative  Efficiency  Frontier  for  MOE  1  (High  Red  Stealth). 
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(2)  MOE  2  vs.  Relative  Cost  (High  Stealth) 

Figure  26  represents  the  MOE  2  DP  for  high  red  stealth  with  a  relative  total 
system  cost  value.  DP  176  has  the  least  relative  cost  and  satisfies  both  MOEs  with  an 
MOE  2  value  of  2.69  hours.  Nonetheless,  DP  241  can  satisfy  both  MOEs  and  provide  a 
better  MOE  2  value  of  1 .99  hours. 


MOE  2  High  Red  Stealth  DP's 

18 

A 

A 

16 


Total  cost 

Figure  26.  Relative  Efficiency  Frontier  for  MOE  2  (High  Red  Stealth). 

(3)  High  Stealth  Design  Points  Relative  Cost 

Table  1 1  represents  all  design  points  for  the  high  red  stealth  scenario  and  its 
relative  cost.  DP  176  is  the  low  relative  cost  option  if  the  GCF  decided  to  account  for 
highly  stealthy  enemies.  However,  other  design  point  options,  such  as  DP  241  provide  a 
higher  percentage  of  detection  and  less  time  to  detect  40%  of  the  red  agents  when  red 
stealth  is  high.  Although  it  is  relatively  more  expensive  than  DP  176,  the  added  benefits 
of  DP  241  may  justify  the  cost. 
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Table  1 1 .  High  Red  Stealth  DP  Options  Relative  Cost. 


DP 

Num 

UAV 

UAV 

Speed 

UAV 

Altit 

UAV 

Max 

Detec 

Range 

Time 

Bw 

Detec 

Endur¬ 

ance 

Refuel 

Time 

Slew 

Rate 

Aper¬ 

ture 

Red 

Stealth 

MOE2 

time 

To 

Detect 

40% 

(Hr) 

MOE1 

%Red 

Agent 

Detected 

Relative 

Cost 

176 

3 

106 

5266 

2734 

7 

6 

31 

346 

120 

75 

2.69 

0.9028 

$  1,577,786.87 

211 

4 

183 

5188 

2172 

7 

6 

60 

294 

118 

77 

2.36 

0.9207 

$  2,049,055.26 

221 

4 

181 

3469 

3250 

17 

5 

57 

158 

117 

80 

2.58 

0.9004 

$  1,925,269.63 

226 

4 

185 

6281 

3813 

18 

6 

49 

332 

100 

78 

2.68 

0.9072 

$  2,146,678.15 

227 

4 

140 

5500 

2500 

13 

5 

45 

210 

90 

75 

2.53 

0.91465 

$  1,867,307.31 

241 

4 

106 

5266 

2734 

7 

6 

31 

346 

120 

75 

1.99 

0.9324 

$  2,103,715.83 

246 

4 

112 

6047 

2313 

16 

3 

33 

351 

96 

74 

2.45 

0.9093 

$  1,755,737.39 

e.  Design  Points  Mission  Success  Probability 

MOE  1  and  2  data  for  each  design  point  resulted  from  averaging  40  runs  of  this 
DP,  as  discussed  in  the  section  on  the  number  of  replications.  For  each  design  point  that 
was  successful  in  meeting  the  threshold  of  both  MOEs,  we  ran  the  replication  an 
additional  100  times.  This  allows  calculating  the  proportion  of  those  runs  that  effectively 
meet  both  MOEs’  mission  success  (see  Appendix  G). 

We  calculated  a  95%  confidence  interval  of  the  proportion  over  the  100  results  for 
each  DP.  The  lower  Cl  value  defines  the  conservative  estimate  of  a  DP’s  probability  of 
mission  success. 

In  Figures  27  and  28,  we  plotted  both  scenarios’  lower  level  of  the  95%  Cl  for  the 
100  runs.  Figure  27  shows  that  the  low  red  stealth  DP  79  has  a  97.04%  probability  of 
mission  success.  It  also  shows  that  there  are  multiple  DP  options  that  achieved  a  100% 
probability  of  mission  success.  The  cheapest  DP  for  the  low  red  stealth  scenario  that 
scores  100%  is  DP  205.  Although  it  relatively  costs  $705,000  USD  more  than  DP  79,  it 
increases  mission  success  by  only  3%. 


55 


Figure  27.  Low  Red  Stealth  Mission  Success. 


The  observation  regarding  the  high  red  stealth  DPs  in  Figure  28  shows  that  the 
most  cost-effective  option  for  high  red  stealth,  DP  176,  has  a  low  probability  of  mission 
success,  at  only  44.18%.  This  indicates  that  if  this  option  is  chosen  by  the  GCF,  it  has  a 
low  probability  of  delivering  the  required  MOE  thresholds.  The  other  DP  option  for  the 
high  red  stealth  scenario  is  DP  241;  this  DP  option  will  provide  100%  probability  of 
mission  success. 


56 


High  Red  Mission  Success 


Figure  28.  High  Red  Stealth  Mission  Success. 
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V.  CONCLUSION 


This  research  examines  how  the  addition  of  UAVs  in  the  detection  operation 
along  the  border  of  Yemen  and  Saudi  Arabia  can  help  improve  the  low  detection  rates  of 
infiltrators.  Findings  from  this  study  provide  useful  information  for  decision  makers 
seeking  suitable  acquisition  options  for  an  appropriate  UAV  to  support  GCF  ground 
forces  in  the  detection  operation. 

To  provide  support  for  the  GCF  on  how  to  defend  their  operational  areas  against 
enemy  incursions  and  reconnaissance,  we  examined  the  role  of  UAV  technology  for  the 
Decisive  Storom  operation  sin  Yemen.  Using  a  systems  engineering  mindset  and 
applying  modeling  and  simulation,  aided  in  providing  insights  to  the  detection 
capabilities  that  Gulf  Coalition  Force’s  desires.  The  experimental  design  helped 
determine  the  optimal  technical  characteristics  for  a  UAV  that  can  enhance  the  ISR  role 
in  the  operation  area.  Regression  and  tree  analysis  tools  were  used  for  exploring  and 
examining  all  design  points  in  this  study.  Finally,  the  author  conducted  a  comparative 
cost  analysis  to  identify  the  desirable  option  GCF  wish  to  pursue  for  both  cases  in  this 
study.  These  outcomes  can  help  the  GCF  decision  makers  with  their  acquisition  of  UAVs 
for  the  Decisive  Storm  operations  in  Yemen. 

A.  PRIMARY  FINDINGS 

•  The  current  detection  capability  of  the  GCF  is  insufficient  in  terms  of 
detection  frequency  and  time  of  detection.  This  situation  drives  the  GCF’s 
need  for  enhanced  detection  methods  to  protect  the  border  more 
efficiently. 

•  Introducing  UAVs  to  support  the  GCF  operation  can  boost  the  percentage 
of  enemy  detections  that  the  GCF  can  achieve.  Despite  the  number  of 
UAV  sets  employed  in  the  simulations  described  in  this  research,  the 
addition  of  this  technology  made  a  49%  increase  in  the  percentage  of 
enemy  detections  over  the  baseline  model  configuration,  which  employed 
no  UAVs.  The  time  to  detect  infiltrators  also  showed  promising  results 
with  the  inclusion  of  UAVs  in  the  detection  operation  simulations. 

•  The  introduction  of  UAVs  into  the  detection  operation  improved  the 
enemy  detection  rate;  however,  the  UAVs  failed  to  satisfy  the  operational 
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thresholds  of  both  MOEs  because  the  vehicles  did  not  have  the  right 
capabilities. 

•  To  minimize  financial  obligations  and  maximize  cost  efficiency,  it  is 
essential  that  decision  makers  select  the  right  UAV  group  for  the 
operation,  as  well  as  the  optimal  quantity  of  UAV  sets  required  for  the 
mission. 

•  Higher  capability  UAVs  will  provide  the  required  operational  goals  and 
might  even  exceed  them,  but  this  option  can  lead  to  a  substantial  financial 
investment  and  surplus  flight  performance  and  sensor  capabilities. 

•  The  relative  cost  analysis  section  of  this  research  reveals  the  desirable 
design  point  option  for  both  case  scenarios.  Both  design  points  for  high 
and  low  red  stealth  scenarios  have  flight  performance  capabilities  that  fall 
into  the  Group  3  UAV  category  as  described  by  the  U.S.  Anny  UAS  Road 
Map  2010-2035  (Army  UAS  CoE  2010). 

•  The  referenced  UAV  for  this  study  was  from  the  Group  1  UAV  category 
as  described  by  the  U.S.  Anny  UAS  Road  Map  2010-2035  (Anny  UAS 
CoE  2010);  however,  the  best  design  points  show  a  need  to  procure  a 
Group  3  UAV  to  perform  the  detection  operation  on  the  Saudi- Yemeni 
border. 

B.  OTHER  IMPORTANT  FINDINGS 

•  A  number  of  possible  DPs  can  meet  the  threshold  levels  for  both  MOEs  in 
either  red  stealth  capability  environment. 

•  The  regression  analysis  discovered  the  factors  that  have  the  most 
significant  effect  on  the  MOEs.  For  the  percentage  of  red  agents  detected, 
it  specified  that  the  red  stealth  level,  number  of  UAVs,  UAV  maximum 
detection  range,  aperture  angle,  and  the  UAV  endurance  all  have  an  effect 
on  MOE  1.  Additionally,  the  red  stealth  level,  number  of  UAVs,  UAV 
maximum  detection  range,  aperture  angle,  UAV  endurance,  and  UAV 
speed  have  an  effect  on  the  time  it  takes  to  detect  40%  of  the  red  agents. 
The  regression  analysis  also  shows  that  the  red  stealth  level  is  a  major 
factor  in  the  detection  of  red  agents. 

•  The  partition  tree  analysis  helped  to  identify  the  most  suitable  UAV 
factors  for  achieving  the  operational  goals  of  the  GCF  in  both  extreme 
cases  (high  and  low  red  stealth  environment).  Both  cases  have  been 
broken  down  and  the  number  of  UAVs  is  a  major  factor  in  detection  for 
both  cases.  Few  design  points  showed  that  fewer  than  four  UAV  sets  are 
able  to  meet  both  MOEs  operational  goals. 
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•  The  partition  tree  analysis  showed  that  solutions  using  more  UAVs  tend  to 
give  better  results  or  more  options  to  achieve  the  MOEs’  threshold,  but  the 
marginal  improvement  after  the  third  UAV  set  does  not  appear  to  justify 
the  cost  in  the  low  red  stealth  case. 

•  The  relative  cost  analysis  discovered  the  desirable  UAV  factor 
combination  for  both  case  scenarios.  Specifically,  DP  79  was  the  desired 
option  for  the  low  red  stealth  scenario.  It  has  the  highest  percentage  of  red 
agent  detection  and  it  has  a  high  probability  of  mission  success. 

•  The  relative  cost  analysis  also  discovered  the  desirable  UAV  factor 
combination  for  the  high  red  stealth  scenario.  Specifically,  DP  241  was  the 
best  design  point  option  with  the  highest  percentage  of  red  agent  detection 
and  lowest  time  to  detect  40%  of  the  red  agents.  It  also  had  a  100% 
probability  of  mission  success.  The  relative  cost  analysis  proved  helpful  in 
identifying  the  DPs  for  the  GCF  in  their  detection  mission. 

•  The  best  UAV  factors  combination  for  the  low  red  stealth  scenario  is  as 
follows:  speed  of  149  knots,  4641  meters  in  altitude,  maximum  detection 
range  of  3344  meters,  19  second  time  between  detections,  endurance  of  4 
hours,  37  minutes  of  refuel  time,  slew  rate  of  201  degrees,  and  an  aperture 
of  1 16  degrees. 

•  The  best  UAV  factors  combination  for  the  high  red  stealth  scenario  is  as 
follows:  speed  of  106  knots,  5266  meters  in  altitude,  maximum  detection 
range  of  2734  meters,  7  second  time  between  detections,  endurance  of  6 
hours,  3 1  minutes  of  refuel  time,  slew  rate  of  346  degrees,  and  an  aperture 
of  120  degrees. 


C.  FUTURE  STUDIES 

As  mentioned  earlier  in  this  thesis,  our  analyses  identified  other  issues  such  as 
route  development  and  UAV  sensor  capabilities  that  merit  more  consideration  but  fell 
outside  the  scope  of  this  research.  The  following  are  potential  areas  to  explore  in  future 
studies: 

•  Use  the  exact  same  type  and  number  of  GCF  detection  equipment  and 
their  technical  parameters  for  future  simulations. 

•  Model  the  impact  of  the  friendly  forces  in  the  operational  area  or  civilians 
by  adding  classification  factors  into  the  UAV. 

•  Study  the  effect  of  false  targets  detections  that  sensors  generate. 
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•  Consider  a  detection  method  that  red  agents  can  use  against  UAVs  and 
allow  them  to  hide  before  they  are  detected  or  cause  damage  to  UAV. 

•  Consider  adding  extra  manned  aircraft,  such  as  F-16s,  in  the  operational 
area  to  support  detection  missions,  to  simulate  the  communication  and 
data  exchange  between  UAVs  and  manned  aircrafts. 

•  Apply  data-link  communication  timings  to  simulate  real  life  scenario 
between  UAVs  and  the  command  centers. 

•  Allow  UAV  to  change  flying  paths  based  on  target  locations. 

•  Use  detailed  data  for  the  reference  UAV  for  the  cost  analysis. 
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APPENDIX  A.  65  DESIGN  POINTS 


UAVSpeed 

UAVAItitude 

UAVMaxDetRng 

UAVMaxTimeBtwDet 

UAVEndurance 

UAVRefuelTime 

UAVSIewRate 

UAVAperture 

Red  Stealth 

1 

166 

3234 

2078 

10 

2 

53 

299 

89 

84 

2 

194 

6594 

1328 

11 

3 

38 

224 

105 

79 

3 

187 

4797 

3859 

8 

3 

56 

107 

87 

73 

4 

157 

7453 

3156 

12 

1 

43 

140 

76 

68 

5 

191 

5344 

1563 

5 

2 

35 

135 

97 

77 

6 

144 

7609 

1703 

12 

2 

50 

102 

112 

83 

7 

172 

3938 

2594 

5 

3 

40 

327 

85 

68 

8 

178 

6828 

3766 

10 

4 

58 

271 

61 

70 

9 

163 

3156 

1047 

17 

4 

48 

177 

66 

81 

10 

196 

6438 

2453 

16 

1 

38 

337 

83 

78 

11 

142 

3078 

3906 

13 

2 

43 

154 

101 

65 

12 

198 

5578 

3063 

19 

2 

32 

173 

109 

71 

13 

146 

4094 

1984 

15 

4 

45 

116 

77 

84 

14 

174 

5734 

2266 

18 

3 

59 

74 

60 

75 

15 

149 

4641 

3344 

19 

4 

37 

201 

116 

68 

16 

161 

5891 

2641 

16 

2 

59 

360 

94 

74 

17 

183 

5188 

2172 

7 

6 

60 

294 

118 

77 

18 

148 

7297 

1797 

10 

5 

44 

308 

108 

72 

19 

168 

4953 

2688 

9 

6 

57 

69 

84 

76 

20 

153 

6672 

3578 

6 

5 

35 

191 

70 

81 

21 

179 

4016 

2219 

6 

8 

41 

65 

102 

71 

22 

176 

7141 

1000 

7 

5 

51 

205 

113 

68 

23 

200 

5031 

3391 

9 

8 

34 

276 

68 

76 

24 

151 

8000 

3484 

11 

6 

46 

233 

73 

85 

25 

155 

4250 

1281 

14 

7 

52 

341 

69 

67 

26 

170 

6125 

1469 

20 

6 

36 

252 

82 

73 

27 

181 

3469 

3250 

17 

5 

57 

158 

117 

80 

28 

189 

7219 

2875 

17 

7 

39 

98 

99 

80 

29 

193 

4484 

1375 

14 

5 

36 

182 

65 

69 

30 

164 

7688 

2031 

16 

7 

60 

163 

92 

66 

31 

159 

3625 

3109 

13 

8 

42 

290 

106 

83 

32 

185 

6281 

3813 

18 

6 

49 

332 

100 

78 

33 

140 

5500 

2500 

13 

5 

45 

210 

90 

75 

34 

114 

7766 

2922 

15 

7 

37 

121 

91 

66 

63 


35 

86 

4406 

3672 

14 

6 

53 

196 

75 

71 

36 

93 

6203 

1141 

17 

6 

34 

313 

93 

77 

37 

123 

3547 

1844 

13 

8 

47 

280 

104 

83 

38 

89 

5656 

3438 

20 

7 

55 

285 

83 

73 

39 

136 

3391 

3297 

13 

7 

40 

318 

68 

67 

40 

108 

7063 

2406 

20 

6 

50 

93 

95 

82 

41 

103 

4172 

1234 

15 

5 

32 

149 

119 

80 

42 

118 

7844 

3953 

8 

5 

42 

243 

114 

69 

43 

84 

4563 

2547 

9 

8 

52 

83 

98 

72 

44 

138 

7922 

1094 

12 

7 

47 

266 

79 

85 

45 

82 

5422 

1938 

6 

7 

58 

248 

71 

79 

46 

134 

6906 

3016 

10 

5 

45 

304 

103 

66 

47 

106 

5266 

2734 

7 

6 

31 

346 

120 

75 

48 

131 

6359 

1656 

6 

5 

53 

219 

64 

82 

49 

119 

5109 

2359 

9 

7 

31 

60 

86 

76 

50 

97 

5813 

2828 

18 

3 

30 

126 

62 

73 

51 

133 

3703 

3203 

15 

4 

46 

112 

72 

78 

52 

112 

6047 

2313 

16 

3 

33 

351 

96 

74 

53 

127 

4328 

1422 

19 

4 

55 

229 

110 

69 

54 

101 

6984 

2781 

19 

1 

49 

355 

78 

79 

55 

104 

3859 

4000 

18 

4 

39 

215 

67 

82 

56 

80 

5969 

1609 

16 

1 

56 

144 

113 

74 

57 

129 

3000 

1516 

14 

3 

44 

187 

107 

65 

58 

125 

6750 

3719 

11 

2 

38 

79 

111 

83 

59 

110 

4875 

3531 

5 

3 

54 

168 

98 

78 

60 

99 

7531 

1750 

8 

4 

33 

262 

63 

70 

61 

91 

3781 

2125 

8 

2 

51 

323 

81 

70 

62 

88 

6516 

3625 

11 

4 

54 

238 

115 

81 

63 

116 

3313 

2969 

9 

2 

30 

257 

88 

84 

64 

121 

7375 

1891 

12 

1 

48 

130 

74 

67 

65 

95 

4719 

1188 

7 

3 

41 

88 

80 

72 

See  text  for  data  collection  methods  and  details. 
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APPENDIX  B.  REQUIRED  RUN  CALCULATION 


A.  MOE 1 


Calculate  sample  size  required  for  desired  Cl  precision 

alpha 

0.05 

Type  l  error  rate 

confidence 

0.95 

=  1- alpha 

z-sub-alpha-over-2 

1.959963985 

(assuming  2-tailed  Cl) 

s  (sample  std  dev) 

8.769529432 

Desired  Width  of  Cl  (twice  the  Margin  of  Error) 

6 

required  sample  size  (n) 

33 

Sample  Target 

Difference  Size  Power  Actual  Power 

6  8  0.47  0.490119 


Power  and  Sample  Size 

1-Sample  Z  Test 

Testing  mean  =  null  (versus  not  =  null) 
Calculating  power  for  mean  =  null  +  difference 
Alpha  =  0.05  Assumed  standard  deviation  =8.77 


Difference 

6 


Sample  Target 

Size  Power  Actual  Power 

33  0.975  0.975591 
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B 


MOE  2 


Method  2:  Calculate  sample  size  required  for  desired  Cl  precision 

alpha 

0.05 

Type  l  error  rate 

confidence 

0.95 

=  1- alpha 

z-sub-alpha-over-2 

1.959963985 

(assuming  2-tailed  Cl) 

s  (sample  std  dev) 

291 

Desired  Width  of  Cl  (twice  the  Margin  of  Error) 

180 

required  sample  size  (n) 

41 

Power  and  Sample  Size 

1-Sample  Z  Test 

Testing  mean  =  null  (versus  not  =  null) 
Calculating  power  for  mean  =  null  +  difference 
Alpha  =0.05  Assumed  standard  deviation  =  291 


Sample  Target 

Difference  Size  Power  Actual  Power 

180  41  0.975  0.977289 


See  text  for  data  collection  methods  and  details. 
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APPENDIX  C.  BLUE  RADAR  DESIGN  POINTS  AND  DETECTION 

PROBABILITIES 


DP 

Num 

Radar 

Range 

Tme  b/w 
Detection 

Std  Dev 

(Alleg2Cas.Red.) 

Mean 

(PercentageRedDetected) 

1 

7 

7313 

60 

8.625543461 

0.4498 

2 

8 

6875 

67 

7.571496821 

0.41885 

3 

7 

7750 

81 

9.532367263 

0.41365 

4 

7 

6500 

69 

8.320348859 

0.4089 

5 

8 

5875 

71 

9.23691035 

0.3795 

6 

8 

7375 

98 

8.80617499 

0.3774 

7 

8 

6750 

89 

9.884947116 

0.37415 

8 

7 

6188 

85 

8.621343521 

0.36665 

9 

7 

7563 

103 

8.689479345 

0.36135 

10 

7 

6313 

93 

8.859443752 

0.3537 

11 

7 

5750 

83 

8.208813275 

0.348 

12 

8 

5188 

68 

7.361542481 

0.3375 

13 

8 

6063 

101 

8.230493645 

0.3349 

14 

7 

7063 

115 

9.272201354 

0.33405 

15 

8 

6625 

116 

8.738215508 

0.3301 

16 

7 

5000 

66 

9.185237599 

0.32825 

17 

7 

5563 

94 

7.445141248 

0.31965 

18 

8 

5625 

108 

8.619447302 

0.3035 

19 

8 

5438 

117 

8.556898552 

0.2872 

20 

7 

4813 

84 

8.318615208 

0.28615 

21 

7 

4125 

61 

9.106070109 

0.2849 

22 

7 

4750 

92 

9.448884075 

0.28095 

23 

7 

4188 

82 

9.508397099 

0.25945 

24 

7 

4500 

102 

8.476059801 

0.2471 

25 

7 

4375 

105 

8.229052661 

0.23995 

26 

6 

7938 

62 

6.842364264 

0.2331 

27 

6 

7688 

74 

6.93814244 

0.20925 

28 

5 

7625 

75 

6.944422222 

0.20665 

29 

6 

7438 

76 

6.535112264 

0.2052 

30 

5 

7500 

78 

7.397851386 

0.2026 

31 

6 

6688 

73 

6.60414594 

0.19755 

32 

5 

7813 

98 

7.08008221 

0.18805 

33 

5 

7250 

88 

7.727365624 

0.18585 

34 

5 

5375 

64 

6.455170855 

0.1857 

35 

6 

8000 

110 

7.259829764 

0.1825 

36 

5 

7188 

96 

6.208141883 

0.1787 

37 

5 

6438 

86 

7.298752967 

0.1768 

38 

6 

6000 

90 

5.917759018 

0.17365 

67 


39 

6 

7125 

100 

6.341560109 

0.1736 

40 

6 

5063 

68 

6.795737729 

0.1717 

41 

5 

7875 

119 

5.648689433 

0.1714 

42 

5 

5688 

87 

4.790749634 

0.1713 

43 

5 

4938 

65 

4.635357425 

0.17095 

44 

5 

6250 

97 

6.299725269 

0.16835 

45 

6 

6938 

112 

5.461050899 

0.1647 

46 

5 

5813 

95 

6.002349967 

0.1633 

47 

6 

7000 

114 

5.457939406 

0.16085 

48 

6 

4875 

80 

6.29183435 

0.1601 

49 

4 

6563 

63 

4.228641656 

0.15725 

50 

5 

4438 

77 

5.679246971 

0.1549 

51 

6 

4000 

70 

6.13418333 

0.1525 

52 

4 

6375 

72 

5.355981559 

0.15135 

53 

5 

5500 

111 

6.339487563 

0.15125 

54 

6 

5313 

107 

6.129740447 

0.14875 

55 

6 

4563 

104 

5.775223383 

0.14135 

56 

6 

4313 

106 

4.955636522 

0.13715 

57 

5 

4250 

99 

4.965973967 

0.13685 

58 

4 

5938 

79 

4.711143364 

0.1368 

59 

5 

4688 

120 

5.496852246 

0.1304 

60 

4 

6813 

113 

5.323147324 

0.1277 

61 

6 

4063 

118 

5.005381719 

0.1253 

62 

4 

6125 

109 

4.933142755 

0.1237 

63 

4 

5250 

91 

4.091814844 

0.12155 

64 

4 

4625 

83 

4.327535569 

0.11425 

65 

4 

5125 

113 

4.291837811 

0.10825 

See  text  for  data  collection  methods  and  details. 
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APPENDIX  D.  DETAILED  PARTITION  TREE  ANALYSIS 


A.  PARTITION  TREE  ANALYSIS  FOR  DESIGN  POINTS  ACHIEVED 

BOTH  MOE’S 

The  partition  tree  in  Figure  29  shows  up  to  three  splits  of  each  MOE’s  options 
using  Red  Stealth  as  the  first  split  factor.  The  Red  Stealth  factor  is  a  non-controllable 
factor,  and  it  depends  on  how  well  the  red  agents  hide  from  detection.  Splitting  the  tree 
according  to  the  Red  Stealth  factor  as  a  first  split  will  assist  the  GCF  in  determining  the 
ideal  UAV  in  both  extreme  cases  for  detecting  red  agents  at  their  most  detectable  (best 
case)  and  least  detectable  (worst  case).  We  will  further  split  the  tree  to  see  how  the  UAV 
factors  contribute  to  success  in  the  highest  and  lowest  levels  of  detectability  cases. 

The  top  box  in  Figure  29  shows  a  count  of  32  DPs  in  the  Red/Blue  bar  out  of  a 
total  of  260  DPs.  This  means  that  32  DPs  accomplished  the  requirement  for  both  MOEs’ 
threshold  by  detecting  90%  of  the  red  agents  within  3.55  hours.  Blue  means  “YES”  and 
red  means  “NO.” 


Figure  29.  Partition  Tree  for  DPs  Achieved  Both  MOEs. 
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Splitting  the  260  design  points  using  the  Red  Stealth  factor  resulted  in  96  DPs 
(top  left  box)  having  a  red  stealth  value  of  less  than  73  (Red  Stealth<73)  among  all  260 
DPs.  Twenty-five  of  those  DPs  only  can  achieve  the  detection  of  90%  of  the  red  agents  in 
less  than  3.55  hours,  while  71  of  the  DPs  failed.  On  the  other  side  (top  right  box),  164 
DPs  (top  right  box)  have  a  red  stealth  value  of  higher  than  or  equal  to  73  (Red 
Stealth>=73)  out  of  260  DPs.  Only  seven  DPs  can  achieve  the  detection  of  90%  of  the 
red  agents  in  less  than  or  equal  to  3.55  hours,  while  157  could  not. 

Further  analysis  will  be  broken  down  based  on  best  and  worst  case  red  agent 
concealment  scenarios. 

B.  LOW  RED  STEALTH  SCENARIO 

The  second  split  after  the  Red  Stealth  factor  is  how  many  sets  of  UAVs  can 
satisfy  the  required  criteria  for  both  MOEs.  The  left  side  box  in  Figure  30  represents  a 
possible  48  DP  options  among  the  260  DPs  that  have  three  sets  or  more  than  three  sets  of 
UAVs.  There  are  22  DPs  in  that  box  that  can  achieve  both  MOEs.  A  third  split  in  same 
figure  under  the  Red  Stealth  factor  is  when  the  UAV  sensor  can  detect  a  maximum  range 
of  more  than  or  equal  to  1797  meters;  22  DPs  out  of  32  DPs  are  possible  options.  Finally, 
a  fourth  split  is  to  check  whether  the  aperture  angle  width  is  greater  than  or  equal  to  91 
degrees.  Nineteen  DPs  are  in  the  blue  area,  which  means  19  DPs  can  achieve  the  best 
case  MOE  1  and  2  thresholds  if  the  UAV  aperture  angle  width  is  greater  than  or  equal  to 
91  degrees  and  the  sensor  maximum  detection  range  is  greater  than  1797  meters. 


70 


Figure  30.  Low  Red  Stealth  Partition  Tree. 


There  is  another  path  to  success  with  fewer  UAV  sets  if  the  GCF  decided  to  go 
for  a  low  number  of  UAVs  solution.  Looking  at  the  right  side  portion  of  the  second  split 
will  detennine  what  capabilities  are  required  to  achieve  the  best  MOE  1  and  2  criteria. 
The  right  side  shows  that  with  fewer  than  three  UAVs,  there  are  three  DPs  than  can 
achieve  the  best  MOE  1  and  2,  while  45  out  of  the  48  DPs  could  not. 

If  the  fewer  than  three  UAV  sets  path  was  selected,  a  third  split  shows  three  DPs 
out  of  24  in  the  blue  area,  which  means  that  there  should  be  at  least  two  or  more  UAV 
sets  to  satisfy  the  best  case  MOE’s  criteria;  fewer  than  two  UAVs  is  not  a  possible 
option.  Further  splitting  shows  that  UAV  aperture  is  required  to  be  greater  than  or  equal 
to  103  degrees  for  fewer  than  two  UAVs  to  satisfy  the  best  case  MOEs.  Three  DPs  are  in 
the  blue  area  if  the  aperture  is  greater  than  103  degrees. 
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C.  HIGH  RED  STEALTH  SCENARIO 

Figure  31  shows  having  a  red  stealth  value  of  at  least  73  resulted  in  164  DPs 
among  all  260  DPs.  Only  seven  DPs  met  the  criteria  of  being  capable  of  detecting  90%  of 
the  red  agents  in  3.55  hours  or  less.  This  shows  the  higher  the  red  stealth  value,  the  harder 
it  is  to  detect  red  agents  or  the  fewer  options  we  have  to  be  able  to  detect  them. 

The  second  split  is  how  many  UAV  sets  can  satisfy  the  required  criteria  for  both 
MOEs.  The  box  on  the  left  side  presents  a  possible  41  DP  options  among  the  260  DPs 
that  have  at  least  four  sets  of  UAVs.  There  are  six  DPs  in  that  box  in  blue,  which  means 
that  those  DPs  are  possible  options  to  achieve  both  MOEs’  criteria. 

A  third  split  under  the  Red  Stealth  factor  is  to  check  whether  the  aperture  angle 
width  is  at  least  90  degrees;  the  same  six  DPs  from  the  earlier  split  are  the  only  options. 
Anything  with  less  than  a  90-degree  aperture  is  not  an  option.  Finally,  a  fourth  split 
occurs  when  the  UAV  sensor  can  detect  a  maximum  range  of  at  least  2172  meters.  Again, 
the  same  six  DPs  fall  into  this  UAV  capability  and  anything  less  than  2172  meters  sensor 
detection  is  not  an  option.  This  means  the  MOE  1  and  MOE  2  threshold  is  achievable 
when  red  stealth  is  high  if  at  least  four  UAV  sets  are  used,  sensor  maximum  detection 
range  is  at  least  2172  meters,  and  the  aperture  angle  width  is  at  least  90  degrees. 
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Figure  31.  High  Red  Stealth  Partition  Tree 


If  there  is  a  requirement  to  reduce  the  number  of  UAV  sets,  while  using  high 
value  for  red  stealth,  only  one  possible  DP  option  can  achieve  the  MOE  1  and  MOE  2 
threshold.  The  box  on  the  right  shows  a  total  of  123  DPs  available  for  fewer  than  four 
UAVs  among  the  260  DPs;  only  one  is  in  the  blue  area.  Further  splitting  this  box  to 
explore  what  is  the  single  option  that  can  satisfy  the  requirement  yields  a  sensor 
capability  having  an  aperture  of  at  least  119  degrees.  Only  one  DP  out  of  five  in  this  box 
falls  into  the  blue  area. 
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APPENDIX  E.  ANALYTIC  HIERARCHY  PROCESS 


A.  PAIRWISE  COMPARISON  AND  COLUMN  SUM 


UAVSpeed 

UAVAltit 

UAVMaxDetecRange 

TimeBwDetec 

Endurance 

RefuelTime 

SlewRate 

Aperture 

UAVSpeed 

1 

1/3 

1/4 

3 

1/5 

2 

1/2 

1/6 

UAVAltit 

3 

1 

1/2 

5 

1/3 

4 

2 

1/4 

UAVMaxDetecRange 

4 

2 

1 

6 

1/2 

5 

3 

1/3 

TimeBwDetec 

1/3 

1/5 

1/6 

1 

1/7 

1/2 

1/4 

1/8 

Endurance 

5 

3 

2 

7 

1 

6 

4 

1/2 

RefuelTime 

1/2 

1/4 

1/5 

2 

1/6 

1 

1/3 

1/7 

SlewRate 

2 

1/2 

1/3 

4 

1/4 

3 

1 

1/5 

Aperture 

6 

4 

3 

8 

2 

7 

5 

1 

column  sum 

21.833333 

11.28333 

7.45 

36 

4.5928571 

28.5 

16.08333 

2.717857 

B.  NORMALIZATION 


UAVSpeed 

UAVAltit 

UAVMaxDetecRange 

TimeBwDetec 

Endurance 

RefulTime 

SlewRate 

Aperture 

UAVSpeed 

0.0458015 

0.029542 

0.033557047 

0.083333333 

0.0435459 

0.07017544 

0.031088 

0.061323 

UAVAltit 

0.1374046 

0.088626 

0.067114094 

0.138888889 

0.0725765 

0.14035088 

0.124352 

0.091984 

UAVMaxDetecRange 

0.1832061 

0.177253 

0.134228188 

0.166666667 

0.1088647 

0.1754386 

0.186528 

0.122646 

TimeBwDetec 

0.0152672 

0.017725 

0.022371365 

0.027777778 

0.0311042 

0.01754386 

0.015544 

0.045992 

Endurance 

0.2290076 

0.265879 

0.268456376 

0.194444444 

0.2177294 

0.21052632 

0.248705 

0.183968 

RefulTime 

0.0229008 

0.022157 

0.026845638 

0.055555556 

0.0362882 

0.03508772 

0.020725 

0.052562 

SlewRate 

0.0916031 

0.044313 

0.044742729 

0.111111111 

0.0544323 

0.10526316 

0.062176 

0.073587 

Aperture 

0.2748092 

0.354505 

0.402684564 

0.222222222 

0.4354588 

0.24561404 

0.310881 

0.367937 

C.  SCORES  AND  CONSISTENCY 


UAVSpee 

UAVAltit 

UAVMaxD 

TimeBwDe 

Endurance 

RefulTime 

SlewRate 

Aperture 

Score 

consistency 

UAVSpeed 

0.0458 

0.029542 

0.033557 

0.0833333 

0.0435459 

0.070175 

0.031088 

0.061323 

0.04979578 

8.074073637 

UAVAltit 

0.1374 

0.088626 

0.067114 

0.1388889 

0.0725765 

0.140351 

0.124352 

0.091984 

0.10766222 

8.328310151 

UAVMaxDetecRange 

0.18321 

0.177253 

0.134228 

0.1666667 

0.1088647 

0.175439 

0.186528 

0.122646 

0.15685387 

8.474347312 

TimeBwDetec 

0.01527 

0.017725 

0.022371 

0.0277778 

0.0311042 

0.017544 

0.015544 

0.045992 

0.02416572 

8.157015878 

Endurance 

0.22901 

0.265879 

0.268456 

0.1944444 

0.2177294 

0.210526 

0.248705 

0.183968 

0.22733952 

8.547836835 

RefulTime 

0.0229 

0.022157 

0.026846 

0.0555556 

0.0362882 

0.035088 

0.020725 

0.052562 

0.03401529 

8.07193913 

SlewRate 

0.0916 

0.044313 

0.044743 

0.1111111 

0.0544323 

0.105263 

0.062176 

0.073587 

0.07340364 

8.174082069 

Aperture 

0.27481 

0.354505 

0.402685 

0.2222222 

0.4354588 

0.245614 

0.310881 

0.367937 

0.32676396 

8.507300303 

Cl 

0.04169474 

CR 

0.02957074 

See  text  for  data  collection  methods  and  details. 
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APPENDIX  F.  TOTAL  SYSTEM  RELATIVE  COST  CALCULATION 


1  UAV  set  Cost 

Total  system 
relative  cost 

DP 

No#  of 

UAVs 

1  UAV  cost 

3  UAV  cost 

2  GCS  +  Supp 
Equip 

cost 

0 

1 

$  65,440.97 

$  196,322.92 

$  198,000.00 

$  394,322.92 

$  394,322.92 

1 

1 

$  73,954.14 

$  221,862.43 

$  198,000.00 

$419,862.43 

$419,862.43 

2 

1 

$  76,687.89 

$  230,063.67 

$  198,000.00 

$  428,063.67 

$428,063.67 

3 

1 

$  59,507.63 

$  178,522.90 

$  198,000.00 

$  376,522.90 

$  376,522.90 

4 

1 

$  61,010.15 

$  183,030.46 

$  198,000.00 

$  381,030.46 

$  381,030.46 

5 

1 

$  64,353.94 

$  193,061.82 

$  198,000.00 

$  391,061.82 

$  391,061.82 

6 

1 

$  77,788.33 

$  233,365.00 

$  198,000.00 

$431,365.00 

$431,365.00 

7 

1 

$  88,095.77 

$  264,287.31 

$  198,000.00 

$462,287.31 

$462,287.31 

8 

1 

$  65,718.38 

$  197,155.15 

$  198,000.00 

$  395,155.15 

$  395,155.15 

9 

1 

$  64,767.91 

$  194,303.73 

$  198,000.00 

$  392,303.73 

$  392,303.73 

10 

1 

$  70,751.66 

$  212,254.99 

$  198,000.00 

$410,254.99 

$410,254.99 

11 

1 

$  72,728.67 

$  218,186.00 

$  198,000.00 

$416,186.00 

$416,186.00 

12 

1 

$  71,139.68 

$  213,419.04 

$  198,000.00 

$411,419.04 

$411,419.04 

13 

1 

$  62,219.77 

$  186,659.31 

$  198,000.00 

$  384,659.31 

$  384,659.31 

14 

1 

$  89,870.50 

$  269,611.49 

$  198,000.00 

$467,611.49 

$467,611.49 

15 

1 

$  74,680.66 

$  224,041.97 

$  198,000.00 

$422,041.97 

$422,041.97 

16 

1 

$  104,754.60 

$  314,263.81 

$  198,000.00 

$  512,263.81 

$  512,263.81 

17 

1 

$  95,773.64 

$  287,320.91 

$  198,000.00 

$485,320.91 

$485,320.91 

18 

1 

$  91,223.31 

$  273,669.92 

$  198,000.00 

$471,669.92 

$471,669.92 

19 

1 

$  93,049.80 

$  279,149.41 

$  198,000.00 

$477,149.41 

$477,149.41 

20 

1 

$  106,868.96 

$  320,606.87 

$  198,000.00 

$  518,606.87 

$  518,606.87 

21 

1 

$  88,555.26 

$  265,665.77 

$  198,000.00 

$463,665.77 

$463,665.77 

22 

1 

$  117,294.83 

$  351,884.49 

$  198,000.00 

$  549,884.49 

$  549,884.49 

23 

1 

$  103,741.17 

$  311,223.51 

$  198,000.00 

$  509,223.51 

$  509,223.51 

24 

1 

$  98,743.79 

$  296,231.38 

$  198,000.00 

$494,231.38 

$494,231.38 

25 

1 

$  93,389.22 

$  280,167.65 

$  198,000.00 

$478,167.65 

$478,167.65 

26 

1 

$  94,439.14 

$  283,317.41 

$  198,000.00 

$481,317.41 

$481,317.41 

27 

1 

$  107,161.53 

$321,484.58 

$  198,000.00 

$  519,484.58 

$  519,484.58 

28 

1 

$  77,829.81 

$  233,489.42 

$  198,000.00 

$431,489.42 

$  431,489.42 

29 

1 

$  103,747.47 

$  311,242.40 

$  198,000.00 

$  509,242.40 

$  509,242.40 

30 

1 

$  120,507.79 

$  361,523.36 

$  198,000.00 

$  559,523.36 

$  559,523.36 

31 

1 

$  112,889.85 

$338,669.54 

$  198,000.00 

$  536,669.54 

$  536,669.54 

32 

1 

$  89,608.94 

$  268,826.83 

$  198,000.00 

$466,826.83 

$466,826.83 

33 

1 

$  106,088.45 

$  318,265.35 

$  198,000.00 

$  516,265.35 

$  516,265.35 
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34 

1 

$  97,556.00 

$  292,668.01 

$  198,000.00 

$490,668.01 

$490,668.01 

35 

1 

$  94,841.53 

$  284,524.60 

$  198,000.00 

$482,524.60 

$482,524.60 

36 

1 

$  112,021.79 

$  336,065.37 

$  198,000.00 

$  534,065.37 

$  534,065.37 

37 

1 

$  110,524.60 

$  331,573.81 

$  198,000.00 

$  529,573.81 

$  529,573.81 

38 

1 

$  107,175.48 

$321,526.45 

$  198,000.00 

$  519,526.45 

$  519,526.45 

39 

1 

$  93,742.31 

$  281,226.93 

$  198,000.00 

$  479,226.93 

$479,226.93 

40 

1 

$  83,454.82 

$  250,364.45 

$  198,000.00 

$  448,364.45 

$  448,364.45 

41 

1 

$  105,832.20 

$317,496.61 

$  198,000.00 

$  515,496.61 

$  515,496.61 

42 

1 

$  106,947.90 

$320,843.70 

$  198,000.00 

$  518,843.70 

$  518,843.70 

43 

1 

$  100,777.76 

$  302,333.29 

$  198,000.00 

$  500,333.29 

$  500,333.29 

44 

1 

$  98,847.69 

$  296,543.06 

$  198,000.00 

$494,543.06 

$494,543.06 

45 

1 

$  100,389.74 

$301,169.23 

$  198,000.00 

$499,169.23 

$499,169.23 

46 

1 

$  109,309.65 

$  327,928.96 

$  198,000.00 

$  525,928.96 

$  525,928.96 

47 

1 

$  81,658.93 

$  244,976.79 

$  198,000.00 

$  442,976.79 

$  442,976.79 

48 

1 

$  96,848.77 

$  290,546.30 

$  198,000.00 

$  488,546.30 

$  488,546.30 

49 

1 

$  66,776.04 

$  200,328.12 

$  198,000.00 

$  398,328.12 

$  398,328.12 

50 

1 

$  75,776.95 

$  227,330.85 

$  198,000.00 

$425,330.85 

$425,330.85 

51 

1 

$  80,311.45 

$  240,934.35 

$  198,000.00 

$438,934.35 

$  438,934.35 

52 

1 

$  78,479.62 

$  235,438.86 

$  198,000.00 

$433,438.86 

$433,438.86 

53 

1 

$  64,660.47 

$  193,981.40 

$  198,000.00 

$  391,981.40 

$  391,981.40 

54 

1 

$  82,974.17 

$  248,922.50 

$  198,000.00 

$446,922.50 

$446,922.50 

55 

1 

$  54,419.76 

$  163,259.28 

$  198,000.00 

$  361,259.28 

$  361,259.28 

56 

1 

$  67,788.25 

$  203,364.76 

$  198,000.00 

$401,364.76 

$401,364.76 

57 

1 

$  72,785.63 

$  218,356.89 

$  198,000.00 

$416,356.89 

$416,356.89 

58 

1 

$  78,140.21 

$  234,420.62 

$  198,000.00 

$432,420.62 

$432,420.62 

59 

1 

$  77,090.29 

$  231,270.86 

$  198,000.00 

$429,270.86 

$429,270.86 

60 

1 

$  64,409.49 

$  193,228.48 

$  198,000.00 

$  391,228.48 

$  391,228.48 

61 

1 

$  93,720.78 

$  281,162.34 

$  198,000.00 

$479,162.34 

$479,162.34 

62 

1 

$  67,783.18 

$  203,349.54 

$  198,000.00 

$401,349.54 

$  401,349.54 

63 

1 

$  51,021.64 

$  153,064.91 

$  198,000.00 

$  351,064.91 

$  351,064.91 

64 

1 

$  58,644.91 

$  175,934.73 

$  198,000.00 

$  373,934.73 

$  373,934.73 

65 

2 

$  65,440.97 

$  196,322.92 

$  198,000.00 

$  394,322.92 

$  788,645.84 

66 

2 

$  73,954.14 

$  221,862.43 

$  198,000.00 

$419,862.43 

$  839,724.86 

67 

2 

$  76,687.89 

$  230,063.67 

$  198,000.00 

$  428,063.67 

$  856,127.35 

68 

2 

$  59,507.63 

$  178,522.90 

$  198,000.00 

$  376,522.90 

$  753,045.80 

69 

2 

$  61,010.15 

$  183,030.46 

$  198,000.00 

$  381,030.46 

$  762,060.92 

70 

2 

$  64,353.94 

$  193,061.82 

$  198,000.00 

$  391,061.82 

$  782,123.64 

71 

2 

$  77,788.33 

$  233,365.00 

$  198,000.00 

$431,365.00 

$  862,730.01 

72 

2 

$  88,095.77 

$  264,287.31 

$  198,000.00 

$462,287.31 

$  924,574.63 

73 

2 

$  65,718.38 

$  197,155.15 

$  198,000.00 

$  395,155.15 

$  790,310.31 

78 


74 

2 

$  64,767.91 

$  194,303.73 

$  198,000.00 

$  392,303.73 

$  784,607.46 

75 

2 

$  70,751.66 

$  212,254.99 

$  198,000.00 

$410,254.99 

$  820,509.97 

76 

2 

$  72,728.67 

$  218,186.00 

$  198,000.00 

$416,186.00 

$  832,371.99 

77 

2 

$  71,139.68 

$  213,419.04 

$  198,000.00 

$411,419.04 

$  822,838.08 

78 

2 

$  62,219.77 

$  186,659.31 

$  198,000.00 

$  384,659.31 

$  769,318.63 

79 

2 

$  89,870.50 

$  269,611.49 

$  198,000.00 

$467,611.49 

$  935,222.97 

80 

2 

$  74,680.66 

$  224,041.97 

$  198,000.00 

$  422,041.97 

$  844,083.95 

81 

2 

$  104,754.60 

$314,263.81 

$  198,000.00 

$  512,263.81 

$  1,024,527.63 

82 

2 

$  95,773.64 

$  287,320.91 

$  198,000.00 

$485,320.91 

$  970,641.83 

83 

2 

$  91,223.31 

$  273,669.92 

$  198,000.00 

$471,669.92 

$  943,339.84 

84 

2 

$  93,049.80 

$  279,149.41 

$  198,000.00 

$477,149.41 

$  954,298.81 

85 

2 

$  106,868.96 

$320,606.87 

$  198,000.00 

$  518,606.87 

$  1,037,213.75 

86 

2 

$  88,555.26 

$  265,665.77 

$  198,000.00 

$463,665.77 

$  927,331.55 

87 

2 

$  117,294.83 

$  351,884.49 

$  198,000.00 

$  549,884.49 

$  1,099,768.99 

88 

2 

$  103,741.17 

$311,223.51 

$  198,000.00 

$  509,223.51 

$  1,018,447.02 

89 

2 

$  98,743.79 

$  296,231.38 

$  198,000.00 

$494,231.38 

$  988,462.77 

90 

2 

$  93,389.22 

$  280,167.65 

$  198,000.00 

$478,167.65 

$  956,335.30 

91 

2 

$  94,439.14 

$  283,317.41 

$  198,000.00 

$481,317.41 

$  962,634.82 

92 

2 

$  107,161.53 

$321,484.58 

$  198,000.00 

$  519,484.58 

$  1,038,969.16 

93 

2 

$  77,829.81 

$  233,489.42 

$  198,000.00 

$  431,489.42 

$  862,978.84 

94 

2 

$  103,747.47 

$  311,242.40 

$  198,000.00 

$  509,242.40 

$  1,018,484.79 

95 

2 

$  120,507.79 

$  361,523.36 

$  198,000.00 

$  559,523.36 

$  1,119,046.72 

96 

2 

$  112,889.85 

$  338,669.54 

$  198,000.00 

$  536,669.54 

$  1,073,339.08 

97 

2 

$  89,608.94 

$  268,826.83 

$  198,000.00 

$466,826.83 

$  933,653.66 

98 

2 

$  106,088.45 

$318,265.35 

$  198,000.00 

$  516,265.35 

$  1,032,530.71 

99 

2 

$  97,556.00 

$  292,668.01 

$  198,000.00 

$490,668.01 

$  981,336.03 

100 

2 

$  94,841.53 

$  284,524.60 

$  198,000.00 

$  482,524.60 

$  965,049.20 

101 

2 

$  112,021.79 

$  336,065.37 

$  198,000.00 

$  534,065.37 

$  1,068,130.74 

102 

2 

$  110,524.60 

$331,573.81 

$  198,000.00 

$  529,573.81 

$  1,059,147.62 

103 

2 

$  107,175.48 

$  321,526.45 

$  198,000.00 

$  519,526.45 

$  1,039,052.90 

104 

2 

$  93,742.31 

$  281,226.93 

$  198,000.00 

$479,226.93 

$  958,453.85 

105 

2 

$  83,454.82 

$  250,364.45 

$  198,000.00 

$  448,364.45 

$  896,728.89 

106 

2 

$  105,832.20 

$317,496.61 

$  198,000.00 

$  515,496.61 

$  1,030,993.22 

107 

2 

$  106,947.90 

$  320,843.70 

$  198,000.00 

$  518,843.70 

$  1,037,687.40 

108 

2 

$  100,777.76 

$  302,333.29 

$  198,000.00 

$  500,333.29 

$  1,000,666.57 

109 

2 

$  98,847.69 

$  296,543.06 

$  198,000.00 

$494,543.06 

$  989,086.12 

110 

2 

$  100,389.74 

$  301,169.23 

$  198,000.00 

$499,169.23 

$  998,338.46 

111 

2 

$  109,309.65 

$  327,928.96 

$  198,000.00 

$  525,928.96 

$  1,051,857.92 

112 

2 

$  81,658.93 

$  244,976.79 

$  198,000.00 

$442,976.79 

$  885,953.57 

113 

2 

$  96,848.77 

$  290,546.30 

$  198,000.00 

$  488,546.30 

$  977,092.59 

79 


114 

2 

$  66,776.04 

$  200,328.12 

$  198,000.00 

$  398,328.12 

$  796,656.23 

115 

2 

$  75,776.95 

$  227,330.85 

$  198,000.00 

$425,330.85 

$  850,661.70 

116 

2 

$  80,311.45 

$  240,934.35 

$  198,000.00 

$438,934.35 

$  877,868.70 

117 

2 

$  78,479.62 

$  235,438.86 

$  198,000.00 

$433,438.86 

$  866,877.73 

118 

2 

$  64,660.47 

$  193,981.40 

$  198,000.00 

$  391,981.40 

$  783,962.79 

119 

2 

$  82,974.17 

$  248,922.50 

$  198,000.00 

$446,922.50 

$  893,845.00 

120 

2 

$  54,419.76 

$  163,259.28 

$  198,000.00 

$  361,259.28 

$  722,518.55 

121 

2 

$  67,788.25 

$  203,364.76 

$  198,000.00 

$401,364.76 

$  802,729.52 

122 

2 

$  72,785.63 

$  218,356.89 

$  198,000.00 

$416,356.89 

$  832,713.78 

123 

2 

$  78,140.21 

$  234,420.62 

$  198,000.00 

$432,420.62 

$  864,841.24 

124 

2 

$  77,090.29 

$  231,270.86 

$  198,000.00 

$429,270.86 

$  858,541.72 

125 

2 

$  64,409.49 

$  193,228.48 

$  198,000.00 

$  391,228.48 

$  782,456.95 

126 

2 

$  93,720.78 

$  281,162.34 

$  198,000.00 

$479,162.34 

$  958,324.68 

127 

2 

$  67,783.18 

$  203,349.54 

$  198,000.00 

$401,349.54 

$  802,699.07 

128 

2 

$  51,021.64 

$  153,064.91 

$  198,000.00 

$  351,064.91 

$  702,129.82 

129 

2 

$  58,644.91 

$  175,934.73 

$  198,000.00 

$  373,934.73 

$  747,869.46 

130 

3 

$  65,440.97 

$  196,322.92 

$  198,000.00 

$  394,322.92 

$  1,182,968.75 

131 

3 

$  73,954.14 

$  221,862.43 

$  198,000.00 

$419,862.43 

$  1,259,587.29 

132 

3 

$  76,687.89 

$  230,063.67 

$  198,000.00 

$428,063.67 

$  1,284,191.02 

133 

3 

$  59,507.63 

$  178,522.90 

$  198,000.00 

$  376,522.90 

$  1,129,568.71 

134 

3 

$  61,010.15 

$  183,030.46 

$  198,000.00 

$  381,030.46 

$  1,143,091.39 

135 

3 

$  64,353.94 

$  193,061.82 

$  198,000.00 

$  391,061.82 

$  1,173,185.46 

136 

3 

$  77,788.33 

$  233,365.00 

$  198,000.00 

$431,365.00 

$  1,294,095.01 

137 

3 

$  88,095.77 

$  264,287.31 

$  198,000.00 

$462,287.31 

$  1,386,861.94 

138 

3 

$  65,718.38 

$  197,155.15 

$  198,000.00 

$  395,155.15 

$  1,185,465.46 

139 

3 

$  64,767.91 

$  194,303.73 

$  198,000.00 

$  392,303.73 

$  1,176,911.19 

140 

3 

$  70,751.66 

$  212,254.99 

$  198,000.00 

$410,254.99 

$  1,230,764.96 

141 

3 

$  72,728.67 

$  218,186.00 

$  198,000.00 

$416,186.00 

$  1,248,557.99 

142 

3 

$  71,139.68 

$  213,419.04 

$  198,000.00 

$411,419.04 

$  1,234,257.13 

143 

3 

$  62,219.77 

$  186,659.31 

$  198,000.00 

$  384,659.31 

$  1,153,977.94 

144 

3 

$  89,870.50 

$  269,611.49 

$  198,000.00 

$467,611.49 

$  1,402,834.46 

145 

3 

$  74,680.66 

$  224,041.97 

$  198,000.00 

$422,041.97 

$  1,266,125.92 

146 

3 

$  104,754.60 

$314,263.81 

$  198,000.00 

$  512,263.81 

$  1,536,791.44 

147 

3 

$  95,773.64 

$  287,320.91 

$  198,000.00 

$485,320.91 

$  1,455,962.74 

148 

3 

$  91,223.31 

$  273,669.92 

$  198,000.00 

$  471,669.92 

$  1,415,009.76 

149 

3 

$  93,049.80 

$  279,149.41 

$  198,000.00 

$477,149.41 

$  1,431,448.22 

150 

3 

$  106,868.96 

$  320,606.87 

$  198,000.00 

$  518,606.87 

$  1,555,820.62 

151 

3 

$  88,555.26 

$  265,665.77 

$  198,000.00 

$463,665.77 

$  1,390,997.32 

152 

3 

$  117,294.83 

$  351,884.49 

$  198,000.00 

$  549,884.49 

$  1,649,653.48 

153 

3 

$  103,741.17 

$311,223.51 

$  198,000.00 

$  509,223.51 

$  1,527,670.53 

80 


154 

3 

$  98,743.79 

$  296,231.38 

$  198,000.00 

$494,231.38 

$  1,482,694.15 

155 

3 

$  93,389.22 

$  280,167.65 

$  198,000.00 

$478,167.65 

$  1,434,502.95 

156 

3 

$  94,439.14 

$  283,317.41 

$  198,000.00 

$481,317.41 

$  1,443,952.23 

157 

3 

$  107,161.53 

$  321,484.58 

$  198,000.00 

$  519,484.58 

$  1,558,453.74 

158 

3 

$  77,829.81 

$  233,489.42 

$  198,000.00 

$431,489.42 

$  1,294,468.26 

159 

3 

$  103,747.47 

$  311,242.40 

$  198,000.00 

$  509,242.40 

$  1,527,727.19 

160 

3 

$  120,507.79 

$  361,523.36 

$  198,000.00 

$  559,523.36 

$  1,678,570.08 

161 

3 

$  112,889.85 

$338,669.54 

$  198,000.00 

$  536,669.54 

$  1,610,008.61 

162 

3 

$  89,608.94 

$  268,826.83 

$  198,000.00 

$466,826.83 

$  1,400,480.48 

163 

3 

$  106,088.45 

$  318,265.35 

$  198,000.00 

$  516,265.35 

$  1,548,796.06 

164 

3 

$  97,556.00 

$  292,668.01 

$  198,000.00 

$490,668.01 

$  1,472,004.04 

165 

3 

$  94,841.53 

$  284,524.60 

$  198,000.00 

$482,524.60 

$  1,447,573.79 

166 

3 

$  112,021.79 

$  336,065.37 

$  198,000.00 

$  534,065.37 

$  1,602,196.10 

167 

3 

$  110,524.60 

$  331,573.81 

$  198,000.00 

$  529,573.81 

$  1,588,721.42 

168 

3 

$  107,175.48 

$321,526.45 

$  198,000.00 

$  519,526.45 

$  1,558,579.35 

169 

3 

$  93,742.31 

$  281,226.93 

$  198,000.00 

$479,226.93 

$  1,437,680.78 

170 

3 

$  83,454.82 

$  250,364.45 

$  198,000.00 

$  448,364.45 

$  1,345,093.34 

171 

3 

$  105,832.20 

$317,496.61 

$  198,000.00 

$  515,496.61 

$  1,546,489.82 

172 

3 

$  106,947.90 

$320,843.70 

$  198,000.00 

$  518,843.70 

$  1,556,531.10 

173 

3 

$  100,777.76 

$  302,333.29 

$  198,000.00 

$  500,333.29 

$  1,500,999.86 

174 

3 

$  98,847.69 

$  296,543.06 

$  198,000.00 

$  494,543.06 

$  1,483,629.18 

175 

3 

$  100,389.74 

$301,169.23 

$  198,000.00 

$499,169.23 

$  1,497,507.69 

176 

3 

$  109,309.65 

$  327,928.96 

$  198,000.00 

$  525,928.96 

$  1,577,786.87 

177 

3 

$  81,658.93 

$  244,976.79 

$  198,000.00 

$  442,976.79 

$  1,328,930.36 

178 

3 

$  96,848.77 

$  290,546.30 

$  198,000.00 

$  488,546.30 

$  1,465,638.89 

179 

3 

$  66,776.04 

$  200,328.12 

$  198,000.00 

$  398,328.12 

$  1,194,984.35 

180 

3 

$  75,776.95 

$  227,330.85 

$  198,000.00 

$  425,330.85 

$  1,275,992.54 

181 

3 

$  80,311.45 

$  240,934.35 

$  198,000.00 

$438,934.35 

$  1,316,803.05 

182 

3 

$  78,479.62 

$  235,438.86 

$  198,000.00 

$433,438.86 

$  1,300,316.59 

183 

3 

$  64,660.47 

$  193,981.40 

$  198,000.00 

$  391,981.40 

$  1,175,944.19 

184 

3 

$  82,974.17 

$  248,922.50 

$  198,000.00 

$446,922.50 

$  1,340,767.49 

185 

3 

$  54,419.76 

$  163,259.28 

$  198,000.00 

$  361,259.28 

$  1,083,777.83 

186 

3 

$  67,788.25 

$  203,364.76 

$  198,000.00 

$401,364.76 

$  1,204,094.28 

187 

3 

$  72,785.63 

$  218,356.89 

$  198,000.00 

$416,356.89 

$  1,249,070.66 

188 

3 

$  78,140.21 

$  234,420.62 

$  198,000.00 

$  432,420.62 

$  1,297,261.86 

189 

3 

$  77,090.29 

$  231,270.86 

$  198,000.00 

$429,270.86 

$  1,287,812.59 

190 

3 

$  64,409.49 

$  193,228.48 

$  198,000.00 

$  391,228.48 

$  1,173,685.43 

191 

3 

$  93,720.78 

$  281,162.34 

$  198,000.00 

$479,162.34 

$  1,437,487.02 

192 

3 

$  67,783.18 

$  203,349.54 

$  198,000.00 

$401,349.54 

$  1,204,048.61 

193 

3 

$  51,021.64 

$  153,064.91 

$  198,000.00 

$  351,064.91 

$  1,053,194.73 

81 


194 

3 

$  58,644.91 

$  175,934.73 

$  198,000.00 

$  373,934.73 

$  1,121,804.19 

195 

4 

$  65,440.97 

$  196,322.92 

$  198,000.00 

$  394,322.92 

$  1,577,291.67 

196 

4 

$  73,954.14 

$  221,862.43 

$  198,000.00 

$419,862.43 

$  1,679,449.72 

197 

4 

$  76,687.89 

$  230,063.67 

$  198,000.00 

$428,063.67 

$  1,712,254.69 

198 

4 

$  59,507.63 

$  178,522.90 

$  198,000.00 

$  376,522.90 

$  1,506,091.61 

199 

4 

$  61,010.15 

$  183,030.46 

$  198,000.00 

$  381,030.46 

$  1,524,121.85 

200 

4 

$  64,353.94 

$  193,061.82 

$  198,000.00 

$  391,061.82 

$  1,564,247.29 

201 

4 

$  77,788.33 

$  233,365.00 

$  198,000.00 

$431,365.00 

$  1,725,460.02 

202 

4 

$  88,095.77 

$  264,287.31 

$  198,000.00 

$462,287.31 

$  1,849,149.25 

203 

4 

$  65,718.38 

$  197,155.15 

$  198,000.00 

$  395,155.15 

$  1,580,620.61 

204 

4 

$  64,767.91 

$  194,303.73 

$  198,000.00 

$  392,303.73 

$  1,569,214.92 

205 

4 

$  70,751.66 

$  212,254.99 

$  198,000.00 

$410,254.99 

$  1,641,019.94 

206 

4 

$  72,728.67 

$  218,186.00 

$  198,000.00 

$416,186.00 

$  1,664,743.99 

207 

4 

$  71,139.68 

$  213,419.04 

$  198,000.00 

$  411,419.04 

$  1,645,676.17 

208 

4 

$  62,219.77 

$  186,659.31 

$  198,000.00 

$  384,659.31 

$  1,538,637.25 

209 

4 

$  89,870.50 

$  269,611.49 

$  198,000.00 

$467,611.49 

$  1,870,445.94 

210 

4 

$  74,680.66 

$  224,041.97 

$  198,000.00 

$422,041.97 

$  1,688,167.90 

211 

4 

$  104,754.60 

$314,263.81 

$  198,000.00 

$  512,263.81 

$  2,049,055.26 

212 

4 

$  95,773.64 

$  287,320.91 

$  198,000.00 

$485,320.91 

$  1,941,283.65 

213 

4 

$  91,223.31 

$  273,669.92 

$  198,000.00 

$471,669.92 

$  1,886,679.69 

214 

4 

$  93,049.80 

$  279,149.41 

$  198,000.00 

$477,149.41 

$  1,908,597.62 

215 

4 

$  106,868.96 

$320,606.87 

$  198,000.00 

$  518,606.87 

$  2,074,427.50 

216 

4 

$  88,555.26 

$  265,665.77 

$  198,000.00 

$463,665.77 

$  1,854,663.09 

217 

4 

$  117,294.83 

$  351,884.49 

$  198,000.00 

$  549,884.49 

$  2,199,537.97 

218 

4 

$  103,741.17 

$311,223.51 

$  198,000.00 

$  509,223.51 

$  2,036,894.05 

219 

4 

$  98,743.79 

$  296,231.38 

$  198,000.00 

$494,231.38 

$  1,976,925.53 

220 

4 

$  93,389.22 

$  280,167.65 

$  198,000.00 

$  478,167.65 

$  1,912,670.60 

221 

4 

$  94,439.14 

$  283,317.41 

$  198,000.00 

$  481,317.41 

$  1,925,269.63 

222 

4 

$  107,161.53 

$321,484.58 

$  198,000.00 

$  519,484.58 

$  2,077,938.32 

223 

4 

$  77,829.81 

$  233,489.42 

$  198,000.00 

$  431,489.42 

$  1,725,957.68 

224 

4 

$  103,747.47 

$  311,242.40 

$  198,000.00 

$  509,242.40 

$  2,036,969.58 

225 

4 

$  120,507.79 

$361,523.36 

$  198,000.00 

$  559,523.36 

$  2,238,093.44 

226 

4 

$  112,889.85 

$338,669.54 

$  198,000.00 

$  536,669.54 

$  2,146,678.15 

227 

4 

$  89,608.94 

$  268,826.83 

$  198,000.00 

$466,826.83 

$  1,867,307.31 

228 

4 

$  106,088.45 

$  318,265.35 

$  198,000.00 

$  516,265.35 

$  2,065,061.41 

229 

4 

$  97,556.00 

$  292,668.01 

$  198,000.00 

$490,668.01 

$  1,962,672.06 

230 

4 

$  94,841.53 

$  284,524.60 

$  198,000.00 

$482,524.60 

$  1,930,098.39 

231 

4 

$  112,021.79 

$  336,065.37 

$  198,000.00 

$  534,065.37 

$  2,136,261.47 

232 

4 

$  110,524.60 

$331,573.81 

$  198,000.00 

$  529,573.81 

$  2,118,295.23 

233 

4 

$  107,175.48 

$321,526.45 

$  198,000.00 

$  519,526.45 

$  2,078,105.80 
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234 

4 

$  93,742.31 

$  281,226.93 

$  198,000.00 

$479,226.93 

$  1,916,907.71 

235 

4 

$  83,454.82 

$  250,364.45 

$  198,000.00 

$  448,364.45 

$  1,793,457.79 

236 

4 

$  105,832.20 

$  317,496.61 

$  198,000.00 

$  515,496.61 

$  2,061,986.43 

237 

4 

$  106,947.90 

$  320,843.70 

$  198,000.00 

$  518,843.70 

$  2,075,374.80 

238 

4 

$  100,777.76 

$302,333.29 

$  198,000.00 

$  500,333.29 

$  2,001,333.14 

239 

4 

$  98,847.69 

$  296,543.06 

$  198,000.00 

$  494,543.06 

$  1,978,172.23 

240 

4 

$  100,389.74 

$  301,169.23 

$  198,000.00 

$499,169.23 

$  1,996,676.91 

241 

4 

$  109,309.65 

$327,928.96 

$  198,000.00 

$  525,928.96 

$  2,103,715.83 

242 

4 

$  81,658.93 

$  244,976.79 

$  198,000.00 

$  442,976.79 

$  1,771,907.14 

243 

4 

$  96,848.77 

$  290,546.30 

$  198,000.00 

$  488,546.30 

$  1,954,185.19 

244 

4 

$  66,776.04 

$  200,328.12 

$  198,000.00 

$  398,328.12 

$  1,593,312.47 

245 

4 

$  75,776.95 

$  227,330.85 

$  198,000.00 

$425,330.85 

$  1,701,323.39 

246 

4 

$  80,311.45 

$  240,934.35 

$  198,000.00 

$438,934.35 

$  1,755,737.39 

247 

4 

$  78,479.62 

$  235,438.86 

$  198,000.00 

$433,438.86 

$  1,733,755.46 

248 

4 

$  64,660.47 

$  193,981.40 

$  198,000.00 

$  391,981.40 

$  1,567,925.58 

249 

4 

$  82,974.17 

$  248,922.50 

$  198,000.00 

$446,922.50 

$  1,787,689.99 

250 

4 

$  54,419.76 

$  163,259.28 

$  198,000.00 

$  361,259.28 

$  1,445,037.11 

251 

4 

$  67,788.25 

$  203,364.76 

$  198,000.00 

$401,364.76 

$  1,605,459.04 

252 

4 

$  72,785.63 

$  218,356.89 

$  198,000.00 

$416,356.89 

$  1,665,427.55 

253 

4 

$  78,140.21 

$  234,420.62 

$  198,000.00 

$432,420.62 

$  1,729,682.48 

254 

4 

$  77,090.29 

$  231,270.86 

$  198,000.00 

$429,270.86 

$  1,717,083.45 

255 

4 

$  64,409.49 

$  193,228.48 

$  198,000.00 

$  391,228.48 

$  1,564,913.91 

256 

4 

$  93,720.78 

$  281,162.34 

$  198,000.00 

$479,162.34 

$  1,916,649.36 

257 

4 

$  67,783.18 

$  203,349.54 

$  198,000.00 

$401,349.54 

$  1,605,398.14 

258 

4 

$  51,021.64 

$  153,064.91 

$  198,000.00 

$  351,064.91 

$  1,404,259.65 

259 

4 

$  58,644.91 

$  175,934.73 

$  198,000.00 

$  373,934.73 

$  1,495,738.93 

See  text  for  data  collection  methods  and  details. 
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APPENDIX  G.  PROBABILITY  OF  MISSION  SUCCESS 


DP# 

Success 

P  hat 

Lower  Cl 

95% 

Relative  Cost 

0 

79 

99 

0.99 

0.97040 

$935,222.97 

1 

106 

77 

0.77 

0.68710 

$1,030,993.22 

2 

110 

89 

0.89 

0.82836 

$998,338.46 

3 

140 

81 

0.81 

0.73272 

$1,230,764.96 

4 

141 

59 

0.59 

0.49312 

$1,248,557.99 

5 

144 

93 

0.93 

0.87974 

$1,402,834.46 

6 

150 

64 

0.64 

0.54545 

$1,555,820.62 

7 

159 

60 

0.6 

0.50350 

$1,527,727.19 

8 

163 

78 

0.78 

0.69840 

$1,548,796.06 

9 

171 

88 

0.88 

0.81599 

$1,546,489.82 

10 

172 

60 

0.6 

0.50350 

$1,556,531.10 

11 

175 

92 

0.92 

0.86656 

$1,497,507.69 

12 

176 

54 

0.54 

0.44182 

$1,577,786.87 

13 

201 

99 

0.99 

0.97040 

$1,725,460.02 

14 

205 

100 

1 

1.00000 

$1,641,019.94 

15 

206 

100 

1 

1.00000 

$1,664,743.99 

16 

209 

100 

1 

1.00000 

$1,870,445.94 

17 

211 

97 

0.97 

0.93640 

$2,049,055.26 

18 

212 

83 

0.83 

0.75601 

$1,941,283.65 

19 

215 

100 

1 

1.00000 

$2,074,427.50 

20 

221 

52 

0.52 

0.42159 

$1,925,269.63 

21 

224 

100 

1 

1.00000 

$2,036,969.58 

22 

226 

71 

0.71 

0.62061 

$2,146,678.15 

23 

227 

88 

0.88 

0.81599 

$1,867,307.31 

24 

228 

100 

1 

1.00000 

$2,065,061.41 

25 

229 

81 

0.81 

0.73272 

$1,962,672.06 

26 

233 

96 

0.96 

0.92140 

$2,078,105.80 

27 

236 

100 

1 

1.00000 

$2,061,986.43 

28 

237 

99 

0.99 

0.97040 

$2,075,374.80 

29 

240 

100 

1 

1.00000 

$1,996,676.91 

30 

241 

100 

1 

1.00000 

$2,103,715.83 

31 

246 

77 

0.77 

0.68710 

$1,755,737.39 

See  text  for  data  collection  methods  and  details. 
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